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Executive Summary 


The 2018 NAEP Oral Reading Fluency (ORF) study, the first NAEP ORF study since the 2002 study, 
was administered to a nationally representative sample of over 1,800 fourth-graders from 180 public 
schools. Oral reading fluency was defined as the ability to read text aloud with speed, accuracy, 
and proper expression. 


Students first completed the NAEP fourth-grade reading assessment and then read four short 
passages out loud to assess their oral reading fluency. Also, to understand the likely underlying 
sources of poor fluency—inefficient word recognition and phonological decoding—students were 
timed and scored for accuracy as they read lists of words and pseudowords (e.g., jad). Pseudowords 
were used to demonstrate students’ ability to phonologically decode unfamiliar words. 


Fourth-graders performing below NAEP Basic were evenly divided into three subgroups (below 
NAEP Basic Low, Medium, and High) based on the distribution of NAEP reading scale scores. This 
is the first time that students performing below NAEP Basic have been grouped in this manner, 
allowing a closer examination of the variation within this important group. 


Key Findings 
©. Across all of the data described in this report, there was a consistent and positive relationship 


between NAEP reading assessment performance and oral reading fluency and between NAEP 
reading assessment performance and word and pseudoword reading. 


O Oral reading fluency, word reading, and pseudoword reading skills declined at each step 
down the NAEP reading achievement levels, from NAEP Advanced to NAEP Proficient 
to NAEP Basic, and then again for each step down through the three below NAEP Basic 
subgroups—below NAEP Basic High, Medium, and Low. 


©. There were noticeable variations in performance among the below NAEP Basic subgroups 
for every dimension of oral reading fluency (reading with proper speed, accuracy, and 
expression) and for each of two “foundational skills,” recognizing familiar written words 
and decoding unfamiliar ones. 


© For fourth-graders in the below NAEP Basic subgroups, particularly those in the below NAEP 
Basic Low subgroup, fluent reading of connected text, such as paragraphs and passages, was 
challenging. 


© About 27 percent of White, 51 percent of Black, and 46 percent of Hispanic fourth-graders 
fell into the below NAEP Basic group. Black students were also overrepresented in the below 
NAEP Basic Low subgroup. 


For a brief version of this report, see its companion publication, Highlights of the 2018 NAEP 
Oral Reading Fluency Study (White et al. 2021), which is available at https://nces.ed.gov/ 


nationsreportcard/studies/orf/. 
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Introduction 


The 2018 National Assessment of Educational Progress (NAEP) Oral Reading Fluency (ORF) study 
was conducted to examine the oral reading performance of fourth-grade public school students. 
The study was administered to a nationally representative sample of 1,800 students between 
January and March of 2018. It measured students’ oral reading fluency in terms of speed, accuracy, 
and expression. 


The NAEP ORF study was administered by the National Center for Education Statistics (NCES) 
first in 1992 and later in 2002. In several ways, the rationale and aims of the previous studies are 
reflected in the current study. As described by Daane et al. (2005), the study was “designed to 
examine several important components of literacy development that are integrated in proficient 
reading—reading accuracy, reading rate, and reading fluency—and to compare these to overall 
reading comprehension as measured in the main NAEP Reading Assessment.” (p. 2). These aims 
will be revisited in this report. 


NCES administers the main NAEP reading assessment (hereafter referred to as NAEP reading 
assessment) every 2 years to students at grade 4. The NAEP reading assessment only assesses 
passage comprehension. In recent years, the results of this assessment have shown that a large 
portion of public-school students performed below NAEP Basic in reading (32 percent in 2015, 
33 percent in 2017, and 35 percent in 2019).' In this report, we focus attention on the oral reading 
fluency, word reading, and pseudoword reading skills of the students performing below NAEP 
Basic. To provide a more nuanced picture, we report results for three subgroups of students 
scoring below NAEP Basic—below NAEP Basic Low, Medium, and High. 


There are no comparisons of results between the current and earlier NAEP ORF studies in this 
report because the design of the 2018 NAEP ORF study has been improved in several important 
ways (and it does not contain the same tasks to allow a link back to the previous NAEP ORF 
studies). First, the current ORF was administered using the new e-NAEP platform on tablets? 
to small groups of students at the same time in a single room, rather than with one-on-one 
administration. Second, the 2018 NAEP ORF student responses were scored for accuracy and 
rate utilizing state-of-the-art, natural language processing scoring algorithms based on spoken 
language processing (see the NAEP ORF website for administrative details, https://nces.ed.gov/ 
nationsreportcard/studies/orf/. 


Finally, in the earlier studies, students had another chance to read the passage silently before 
they read it aloud (as part of the NAEP reading assessment). In the current study, students read 
aloud a set of four passages that they had not seen in the NAEP reading assessment session. This 
“cold read” procedure is now common practice in oral reading fluency probes used in school- 
based screening and progress monitoring assessments (Wayman et al. 2007). The expression scale 
has also been refined in terms of levels and description in consideration of recent developments 


1 See the Nation’s Report Card at https://www.nationsreportcard.gov/reading/nation/achievement/?grade=4 for additional details about 
the grade 4 NAEP reading achievement level. 
* See the NAEP digitally based assessments at https://nces.ed.gov/nationsreportcard/dba/. 
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in the literature in this area (e.g., Kuhn, Schwanenflugel, and Meisinger 2010; Schwanenflugel 
and Benjamin 2017). Finally, the 2018 NAEP ORF study oversampled low-performing students 
in order to provide a more nuanced picture of the fourth-graders in the below NAEP Basic level 
of achievement. 


Importance of Measuring Oral Reading Fluency 


Students who have appropriate speed, accuracy, and expression when reading aloud (i.e., students 
who have oral reading fluency) are more likely to comprehend connected text (Fuchs et al. 2001; 
Sabatini, Wang, and O’Reilly 2019). This is because they are able to conserve cognitive resources 
that can be applied to comprehension of meaning. More specifically, because they have developed 
high-quality lexical representations, they can retrieve word meanings quickly (Perfetti 2007). 


Most administrators and teachers understand that speed, accuracy, and expression when reading 
aloud are essential in support of reading comprehension and learning in subject areas. Educators 
also understand that ORF is a strong indicator of overall reading competence (Fuchs et al. 2001). 
Therefore, ORF has become the primary measurement technique for determining which students 
may be on track toward meeting state reading standards and which students would benefit from 
additional services and intervention (Crawford, Tindal, and Stieber 2001; McGlinchey and Hixson 
2004; Reschly et al. 2009; Silberglitt and Hintze 2005; Stage and Jacobsen 2001). 


Oral reading has been found to be effective in helping students become better readers (Chard, 
Vaughn, and Tyler 2002; Kuhn and Stahl 2003; National Reading Panel 2000; Rasinski and Hoffman 
2003; Samuels and Farstrup 2006; Shanahan 1998; Therrien 2004). In addition, regular classroom 
assessment of oral reading, also known as curriculum-based measurement by some (e.g., Fuchs 
et al. 2001) and progress monitoring by others, has been widely implemented and used to 
place students into different tiers within response-to-intervention models of instruction and 
intervention (Griffiths et al. 2009). 


With so much at stake in terms of the increasing importance of ORF in reading assessments and 
the heightened policy and practice role that it plays in learning accountability systems across 
states, NCES decided to examine again the oral reading performance of a nationally representative 
sample of fourth-graders in 2018. The key idea was to compare several metrics of ORF measures 
to policy-relevant achievement levels from the NAEP reading assessment. 


By providing common profiles of low-fluency fourth-grade readers based on rich evidence from 
the 2018 NAEP ORF data, teachers may be further encouraged to (a) identify students who face 
fluency and foundational skill challenges (i.e., word reading and phonological decoding) and 
(b) create enhanced instructional practices for those readers who are falling behind grade-level 
expectations. The findings may also send an important signal to policymakers and educators: It 
cannot be assumed that all students have achieved oral reading fluency and adequate foundational 
skills by the fourth grade. 
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Purpose of the 2018 NAEP ORF Study 


The main purpose of the 2018 NAEP ORF study is to gain a fuller understanding of how fourth- 
grade students perform on ORF and the relationship of ORF performance to NAEP reading 
assessment performance. Accordingly, this report has the following aims. The first is to explain 
the measures and procedures of this new study. The second is to report on overall results for the 
nation’s fourth-grade public school students and on key group differences, as is typical of NAEP 
reports. The third is to explore the relationship of reading comprehension as measured by the 
NAEP reading assessment to the ORF measures, with special attention to students performing 
below NAEP Basic, which comprise larger percentages of racial/ethnic minority, language and 
linguistic minority, and low socioeconomic status (SES) students. The final aim is to improve 
the interpretation of the NAEP reading assessment scores at the lowest end of the achievement 
spectrum—those below NAEP Basic. 


To anticipate our discussion, most children who score below NAEP Basic on the NAEP reading 
assessment also show various profiles of lower performance on the oral reading measures, which 
emphasizes the importance of word-level accuracy and rate to reading comprehension. Clearly, 
as evidenced in their oral reading, not all children have mastered their foundational skills by 
the fourth grade. Ignoring these problems is not a remedy, as longitudinal studies show that 
poor foundational skills result in stagnant comprehension growth in the middle grades (Wang 
et al. 2019). There is the further issue of student motivation and interest in reading. If reading is 
always effortful and understanding is incomplete, then students will choose to read less, creating 
a vicious circle denying students who need practice and experience with texts the most from 
engaging in reading widely. 


Definition of Oral Reading Fluency 


Following the National Reading Panel’s definition of fluency (National Reading Panel 2000, p. 3-1), 
the 2018 NAEP ORF study operationally defined fluent readers as those who can “read text with 
speed, accuracy, and proper expression.” The theory underlying this operational definition is 
captured by Kuhn and colleagues (2010), who state that fluency is demonstrated during oral 
reading of passages through ease of word recognition and appropriate pacing, phrasing, and 
intonation (i.e., expressiveness), which indicates the extent to which these prosodic features 
match the structure and meaning of the text. 


As can be seen from the above definition, the 2018 NAEP ORF combines accuracy, speed, and 
expression, which, taken together, facilitate the reader’s understanding of textual meaning. 
When there is weakness in oral reading fluency (evidenced by errors in reading, hesitation, or 
inappropriate pauses that signal the student is challenged by the text), it is often explainable 
in terms of the difficulties a child is experiencing in word reading, phonological decoding, 
vocabulary, or grammatical structures of the English language. 
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We call attention here to the rate and accuracy of word reading and phonological decoding 
because they are foundational for fluency and subsequent development of reading comprehension. 
The more words children can recognize in print quickly and without effort, the more efficiently 
they can acquire new vocabulary and knowledge through reading. This, in turn, helps in the 
comprehension of each new text they encounter. 


For children with weak word reading and phonological decoding skills, it is challenging and 
effortful to learn new words from printed sources. Demands for learning subject area vocabulary 
and content knowledge increase across the late elementary, middle, and secondary grades. An 
increasing number of new words must be learned to enable the learning of standards-based factual 
and conceptual content knowledge; more sophisticated comprehension skills and strategies are 
also required to keep pace with content learning demands (McNamara 2007). 
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Method 


Measures 


The 2018 NAEP ORF measures were conceptualized, constructed, and labeled in the following 
manner: 


© Passage reading (Oral Reading Fluency): The ability to read connected text with appropriate 
rate, accuracy, and expression—revealing the ability to use knowledge of language to 
understand texts and draw appropriate inferences. The skills to understand and interpret 
text, in turn, depend on knowledge of vocabulary, sentence structure, sentence meaning, 
genres, and topics, world knowledge, personal experience, and conceptual reasoning. 


© Word reading (also known as word recognition): Following the Literacy Information 
and Communication System (https://lincs.ed.gov), which defines word reading as “the 
ability of a reader to recognize written words correctly and virtually effortlessly,” we 
define word reading as the ability to recognize familiar written words with appropriate 
speed and accuracy—relying primarily on orthographic memory (i.e., where the spelling 
of words triggers memory for meaning and phonology). Children in the primary school 
grades must recognize the printed words they already know when listening or speaking, 
and they must learn to do this with little conscious effort. 


© Phonological decoding: The ability to pronounce a printed sequence of letters based on 
knowledge of spelling-sound correspondences. Unlike word reading, which relies on 
orthographic memory, phonological decoding depends on the association of letter and 
sounds or letter patterns and sound patterns. Many words that are initially pronounced 
by decoding (or “sounding out”) eventually become automatically recognized as words 
with minimal conscious effort. 


Each of the above measures were operationalized in terms of two performance aspects or scores— 
rate and accuracy—as well as the product of the two (words correct per minute). 


© Words correct per minute: The total number of words correctly read divided by the amount 
of time taken to read each passage and word-level lists; that is, the words correct per 
minute (WCPM) score. Note that this variable is dependent on the two measurements, 
rate and accuracy. 


© Accuracy: he percentage of words that was read accurately. For passages, the total 
number of attempted words in the passage was the denominator, and for word lists, the 
total number of words presented to students was the denominator. 


© Rate: The total number of words students attempted to read (i.e., how far in the text they 
reached from the first attempted word) divided by the amount of time taken to read the 
text, yielding a words per minute (WPM) score. Attempted words included words read 
correctly or incorrectly as well as those that were skipped. 
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Passage reading was operationalized in terms of one additional measure—expression—defined 
below: 


© Expression: Appropriate intonation, rhythm, emphasis, and pausing that groups words 
into phrasal and larger units in ways that express the meaning and structure of the text 
and enhance understanding and enjoyment in a listener. 


Description of the Tasks 


The following text materials were given in English to students to be read aloud. See appendix B 
for a more detailed description of stimulus materials and how they were created. 


© Four text passages, each consisting of 152-162 words, providing a measure of fourth- 
graders’ ability to read words aloud in connected texts. 


© Two comparable forms of word lists of 24 English words arranged in increasing order 
of complexity, providing a measure of individual students’ ability to recognize familiar 
words. 


© Two comparable forms of pseudoword lists of 18 non-occurring English forms (e.g., wike), 
providing a pure measure of students’ ability to decode or produce pronunciations of 
words they are unfamiliar with. 


All instructions were given in English and students were expected to provide oral responses in 
English. 


Study Sample and Administration of the Tasks 


A nationally representative sample of 1,800 fourth-graders from 180 public schools participated 
in the 2018 NAEP ORF study with a response rate of 88 percent. Public schools with 75 percent of 
students eligible for the National School Lunch Program (NSLP) were oversampled. Students in 
this sample were weighted to represent the total population of U.S. fourth-grade public school 
students. 


The 2018 NAEP ORF assessment started with a NAEP grade 4 reading assessment tutorial followed 
by the NAEP reading assessment. As is typical in NAEP reading assessments, each student received 
two blocks of items.? Then they responded to the study task module with four ORF feedback 
questions. Lastly, they answered the NAEP fourth-grade reading student questionnaire.’ 


The study task module consisted of four types of tasks in the following order: (1) repeating 
four sentences aloud to measure students’ speaking skills, (2) reading a word list aloud, 
(3) reading a pseudoword list aloud, and (4) reading four passages aloud with a single yes/no 


3 In total, there were 10 blocks of items in the 2017 NAEP reading assessment at grade 4 (the most recent assessment available at the time of 
the study). Two of the 10 blocks were released; therefore, the ORF study used the remaining eight blocks of items to assess students’ reading 
comprehension. 

“ Students’ demographic data were obtained from the NAEP reading student questionnaire. 
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passage meaning question for each passage.° Before students were administered each task type, 
they were shown a tutorial that provided instructions with task-specific examples of a student 
demonstrating each upcoming task. The average time to complete the actual operational ORF 
module was 15 minutes. Figure 1 demonstrates the ORF assessment procedure. 


Figure 1. NAEP Oral Reading Fluency (ORF) study assessment administration 
procedure: 2018 


NAEP Reading Assessment 


NAEP Grade 4 
Reading Assessment 
Welcome Screen and Tutorial 


| 


NAEP Grade 4 
Reading Assessment 


| 
NAEP ORF Module! 


Sentence Repeat Task 


| 


Word Reading Task 


| 


Pseudoword Reading Task 


| ' At the beginning of the NAEP ORF module, students were 
; given general directions for the module including how to 
Passage Reading Task use the headset properly. Then, for each study task, students 


were given task-specific instructions with a demonstration 
before performing the task. For example, for the pseudoword 


ORF Feedback Questions reading task, students were told, “Now, you will read out 
loud from a list of made-up words” and then the tablet screen 

| showed a few examples of made-up words. Four ORF feedback 

questions about the students’ experience in oral reading 

were presented by voice-over with four choices shown on 

NAEP Grade 4 the screen: (1) In this school year, how often have you read 
Sjable(stels Questionnaire out loud in school or at home, or anywhere? (2) Who did you 
read to? (3) Where were you? and (4) How difficult was this 


reading out loud test? 
SOURCE: U.S. Department of Education, Institute of Education 


Sciences, National Center for Education Statistics, National 
Thank You Message Assessment of Educational Progress (NAEP) 2018 Oral Reading 
Fluency study. 


> Each passage was followed by a single yes/no passage meaning question to encourage reading for meaning, but these questions were not 
scored. 
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The 2018 NAEP ORF assessment was administered on the 2018 e-NAEP platform on tablets, 
which presented text and tasks on the screen and recorded the student’s reading aloud. Student 
recordings were transferred from the tablets to a NAEP database for scoring (see “Scoring of 
ORF Measures” below). 


The ORF assessment was administered in groups. Approximately 10 students, spaced at least 
two meters apart, took the assessment in the same room, and students were provided cardboard 
carrels to reduce the noise from the surroundings. Each student wore a headset with a boom- 
mounted noise-canceling microphone to complete the ORF module. The potential impact of 
the number and spacing of multiple students reading aloud on the quality of other students’ 
read-aloud performance and on the suitability of the resulting audio recordings for scoring was 
studied extensively in cognitive laboratory testing and during the feasibility test and pilot study. 
The subsequent operational implementation of the study in 2018 fully addressed all issues and 
concerns identified previously. 


Scoring 


In this study, NCES used a new automatic speech analysis/scoring system’ that calculated accuracy, 
rate, and WCPM variables to score student passage reading recordings. This system transcribed 
the student recordings of passage reading and then aligned the resulting orthographic transcripts 
with the passage text. The system recognizes accepted pronunciations of each word, taking into 
account dialect and second language variations as long as the speaking pattern remains consistent 
throughout the reading. A validation experiment provided evidence that the system can score oral 
readings as well as human scorers do, and that automated scoring is not biased against English 
language learners (ELLs) or Black students (Balogh et al. 2012). In addition, students were not 
penalized for problems unrelated to reading, such as stuttering. 


To evaluate the reliability of the automatic speech analysis system, a sample of the passage 
recordings were transcribed by both the system and a trained human scorer. Each of the two 
transcriptions was aligned with reference to the passage text such that the alignment minimized 
insertions, deletions, and substitutions. Then, within the span of passage text that the student 
attempted to read, the system counted the number of words that were correctly read in the 
correct order. The correlation between the counts of words correctly read using the machine 
transcriptions vs. the human transcriptions of the same recordings was 0.96. 


For recordings of word reading and pseudoword reading, trained human scorers transcribed 
students’ oral responses. Then the analysis/scoring system produced a time alignment of each 
transcript with the corresponding student recording, calculated the duration of the student’s 
list reading, and counted the number of words or pseudowords that were correctly read. These 
counts and the corresponding reading durations were combined to calculate the word reading 
and pseudoword reading WCPM variables. A sample of the students’ oral response recordings 


° See Scoring tab on the NAEP ORF website, https://nces.ed.gov/nationsreportcard/studies/orf/scoring.aspx. 
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was transcribed by two different scorers to evaluate the reliability of human transcripts of the 
two list types. The correlation between the two human transcriptions was 0.99 and 0.97 for word 
reading and pseudoword reading, respectively. 


Passage reading expression was scored by trained human scorers using a 6-point scoring rubric 
(see figure 2). Again, a sample of students’ passage reading responses were scored for expression 
by two scorers to examine human scoring reliability. Between two human scorings, the exact 
agreement rate (i.e., the percentage of scores that were exactly the same) was 58 percent and 
the adjacent agreement (i.e., the percentage of scores that were only one level different) was an 
additional 39 percent.’ For more information about the automatic speech analysis system and 


scoring reliability, see the NAEP ORF website, https://nces.ed.gov/nationsreportcard/studies/orf/. 


The scoring of the passage reading expression measure reflected a holistic judgment that was 
based on the reader’s predominant pattern of performance in expressing structure and meaning. 
Speaking rate within a normal range and word reading accuracy were not considered part of 
expression. Similarly, pronunciation quality was not part of expression. And, in general, if the 
reader skipped a word, inserted a word, or substituted a similar word for a text word, and the 
meaning was reasonably expressed, then the misreading was not counted against expression. 


7 Exact agreement lower than 61 percent was flagged to indicate mild concern; exact agreement lower than 57 percent was flagged to 
indicate greater concern. This criterion was based on the NAEP Writing assessment, which has six scoring categories. See https://nces. 
ed.gov/nationsreportcard/tdw/scoring/scoring_within.aspx for more detail. 
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Figure 2. NAEP Oral Reading Fluency passage reading expression rubric: 2018 


0 Insufficient © Insufficient sample for rating (fewer than 12 words read aloud correctly). 
Sample 

1 Word by © Less than % of the words read aloud with appropriate expression. 
Word © Reading focuses on individual words (not phrases, sentences, or the passage). 


° Reading is all or mostly monotone. 


2 Local ° More than % and less than ¥% of the words read aloud with appropriate 
Grouping expression. 
© Reading focuses on local word groups (with little to no focus on phrases, 
sentences, or the passage). 
© Reading may be mostly arrhythmic or monotone. 


3 Phrase & © More than ¥% of the words read aloud with appropriate expression. 
Clause © Reading expresses the structure or meaning of words, phrases, clauses, and a 
few sentences (with little or no focus on the passage). 
° Intonation may sometimes reinforce rhythmic grouping, or reading may be 


monotone. 
4 Sentence © More than % of the words read aloud with appropriate expression. 
Prosody © Reading correctly expresses text and sentence structure and meaning (which 


may include non-local text connections). 

o Reading can be occasionally inconsistent, but not monotone. 

© Reading rate is at least 55 words per minute (at least 80 text-words-read to 
merit this level or above). 


5 Passage o Passage read as if for a listener—of the passage portion read aloud, all or 
Expression nearly all (at least 90 percent) is read with appropriate expression. The 

reading consistently expresses the structure and meaning of sentences, 
paragraphs, and the passage as a whole (which may include non-local text 
connections). 

© Reading may include a few word stumbles or misreading, but it is expressive 
throughout. 

© Reading rate is at least 80 words per minute (at least 120 text-words-read to 
merit this level). 


8 Silent © Recording has audio signal, but no near-field speech from the student. 
Reader o Audible background sounds, breathing, or microphone touching may suggest 
the reader did not speak throughout the recording period. 


9 Anomaly ° Not 0, 1, 2, 3, 4,5, or 8. Not a silent reader, nor any near-field reading aloud. 
© Possibly with off-task or irrelevant speech, evidence of confusion, or 
anything else unexpected, including electronic crackle or dead flat-line 
signal. 
NOTE: Passage reading expression ratings of 8 and 9 were treated as missing as these students’ expression level could not be 
determined because of the quality/content of the audio file. 


SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment 
of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Results 


Average Oral Reading Fluency Performance and 
Foundational Skills 


Average scores for each of the oral reading fluency measures are shown in table 1. Typical U.S. 
fourth-graders read passages at 120 words correct per minute (WCPM) and 126 words per minute 
with an accuracy of 94 percent (which is about 1 out of every 17 words read incorrectly). The 
average expression score was approximately at Level 4 indicating that, overall, these fourth- 
graders could “correctly express text and sentence structure and meaning” of continuous text 
through appropriate intonation, rhythm, emphasis, and pauses despite occasional inconsistency. 


The average WCPM score was 51 for word reading and 22 for pseudoword reading. This indicates 
that phonological decoding is a slower process, as expected, because reading words letter-by-letter 
or in chunks of letters is distinct from the ability to read words as a whole or read continuous text. 


Table 1. Average oral reading performance of U.S. fourth-graders: 2018 


ORF measure | DY=X Yor gh ho} a Loyal Average 


Oral reading fluency Passage reading words correct per minute (WCPM) 120 
Passage reading rate: Words per minute 126 
Passage reading accuracy: Percentage of words 
read correctly 94 
Passage reading expression 4 
Word reading Word list reading: WCPM 51 
Phonological decoding Pseudoword list reading: WCPM 22 


NOTE: Pseudoword is a made-up but pronounceable word. 
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


Average Oral Reading Fluency Performance by 
Demographic Subgroups: Passage Reading WCPM 


The following figures (figures 3 to 7) visually show the comparison of passage reading WCPM 
scores by gender, race/ethnicity, eligibility status for the National School Lunch Program, ELL 
status, and disability status. Comparisons? for all other passage reading measures and foundational 
skills show similar patterns and can be found in appendix tables C-2 to C-6. 


® All comparisons were conducted with an alpha level of 0.05, with multiple pairwise comparison adjustments applied when needed using 
the False Discovery Rate (FDR) procedure. 
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As shown in figure 3, female students significantly outperformed their male peers in passage 
reading (122 vs. 118 WCPM). Among all race and ethnicity groups (see figure 4), the average 
passage WCPM score for White students (128) was significantly higher than that for Black (106) 
and Hispanic (110) students. No significant difference was found between White and Asian 
students (137) or between White students and students of other races/ethnicities (123), including 
students who were American Indian/Alaska Native, Pacific Islander/Native Hawaiian, and of 
Two or more races. 


Figure 3. Average passage reading WCPM by gender: 2018 


Passage Reading WCPM 
160— 


140 
120 
100 
80 
60 
40 


20 


Male Female 


Gender 


* Statistically significant score difference compared to male students, p < .05. 

NOTE: WCPM is an abbreviation for words correct per minute. The positions of the data points in the 
graphics are based on the unrounded numbers. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Statistics, National Assessment of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Figure 4. Average passage reading WCPM by race/ethnicity: 2018 


Passage Reading WCPM 
160 


140— 
120— 
100— 
80 
60 
40— 


20 — 


White Black Hispanic Asian Other 
Race/ethnicity 


* Statistically significant score difference compared to White students, p < .05. All comparisons were conducted with an 
alpha level of 0.05 with White students as a reference group. The False Discovery Rate (FDR) procedure was used to adjust for 
multiple pairwise comparisons between White and four other racial/ethnic groups (i.e., 4 comparisons). 

NOTE: Black includes African American, and Hispanic includes Latino. Race categories exclude Hispanic origin. Other 
includes American Indian/Alaska Native, Pacific Islander/Native Hawaiian, and Two or more races. WCPM is an abbreviation 
for words correct per minute. The positions of the data points in the graphics are based on the unrounded numbers. 
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


The 2018 NAEP Oral Reading Fluency Study| Results 13 


As figure 5 shows, the average passage WCPM score of fourth-graders who were eligible for free/ 
reduced-price lunch under the NSLP (108) was significantly lower than that of students who were 
not eligible (133). The score gap between ELL and non-ELL students was found to be statistically 
significant (see figure 6). The total number of words read correctly per minute by fourth-grade 
ELLs (99) was, on average, 23 words fewer than their counterparts who were not ELLs (123). 
Lastly, the score gap between students with a disability and students without a disability was 
statistically significant (see figure 7) and even greater than the gap between ELL and non-ELL 
students. On average, the passage WCPM score for students with a disability (92) was 31 words 
fewer than that for those without a disability (124). 


Figure 5. Average passage reading WCPM by National School Lunch Program (NSLP) 
eligibility status: 2018 


Passage Reading WCPM 


1607 
140 5 
1205 
100 + 
80 
133 108* 141 
605 
40 
20 4 
0 Ei errr EEE EE EOOCCC___ 


Not eligible Eligible Information not available 
National School Lunch Program (NSLP) eligibility status 


* Statistically significant score difference compared to students not eligible for NSLP, p < .05. All comparisons were conducted 
with an alpha level of 0.05 with students not eligible for NSLP as a reference group. The False Discovery Rate (FDR) procedure 
was used to adjust for multiple pairwise comparisons between students not eligible NSLP and two other NSLP status groups (i.e., 
2 comparisons). 

NOTE: WCPM is an abbreviation for words correct per minute. The positions of the data points in the graphics are based on the 
unrounded numbers. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Figure 6. Average passage reading WCPM by English language learner (ELL) 
status: 2018 


Passage Reading WCPM 
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140 + 
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Not ELL ELL 
English Language Learner (ELL) status 


* Statistically significant score difference compared to non-ELL students, p <.05. 

NOTE: WCPM is an abbreviation for words correct per minute. The positions of the data points in the 
graphics are based on the unrounded numbers. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Statistics, National Assessment of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Figure 7. Average passage reading WCPM by students with disabilities status: 2018 


Passage Reading WCPM 
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Students with disability status 


* Statistically significant score difference compared to students without disabilities, p <. 05. 

NOTE: WCPM is an abbreviation for words correct per minute. The positions of the data points in the 
graphics are based on the unrounded numbers. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Statistics, National Assessment of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


Average Oral Reading Fluency Performance by NAEP 
Reading Achievement Levels: Passage Reading WCPM 


Description of achievement levels 


To better understand the relationship between students’ oral reading skills and their NAEP reading 
achievement levels, it is helpful to understand what the levels measure. One of the ways NAEP 
reports students’ reading performance results is through achievement levels. The NAEP grade 
4 reading achievement levels are set by the National Assessment Governing Board based on the 
collective judgment of a representative panel of teachers, education specialists, and the general 
public. The achievement levels are based on the NAEP reading scale scores, ranging from 0 to 500. 
The current NAEP reading framework provides descriptions for what students should know and 
be able to do at the NAEP Basic, NAEP Proficient, and NAEP Advanced levels, as shown in figure 8. 
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Figure 8. NAEP grade 4 reading achievement-level descriptions 


Achievement |Cut 
| score |Description 


NAEP Basic 208  Fourth-grade students performing at the NAEP Basic level should be 
able to locate relevant information, make simple inferences, and use 
their understanding of the text to identify details that support a given 
interpretation or conclusion. Students should be able to interpret the 
meaning of a word as it is used in the text. 


NAEP 238  Fourth-grade students performing at the NAEP Proficient level should be 

Proficient able to integrate and interpret texts and apply their understanding of the 
text to draw conclusions and make evaluations. 

NAEP 268  Fourth-grade students performing at the NAEP Advanced level should 

Advanced be able to make complex inferences and construct and support their 


inferential understanding of the text. Students should be able to apply their 
understanding of a text to make and support a judgment. 


NOTE: For additional details about the NAEP grade 4 reading achievement-level descriptions, see here. 
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment 
of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


Although below NAEP Basic is not one of the three NAEP reading achievement levels reported by 
NAEP, it is the focus of the 2018 NAEP ORF study. Among other reasons noted in the Introduction 
of this report, over a third of fourth-grade children may still lack the fluency and foundational 
skills necessary to support continued progress in reading comprehension across their school years. 


Percent distribution by NAEP reading achievement levels and selected 
student characteristics 


As table 2 shows, 36 percent of all fourth-graders were in the below NAEP Basic group. About 
27 percent of White, 51 percent of Black, and 46 percent of Hispanic fourth-grade students fell 
into the below NAEP Basic level. In other words, over half of the nation’s Black fourth-graders 
and nearly half of Hispanic fourth-graders performed below NAEP Basic. It is noteworthy that the 
reverse was true at the upper achievement levels. At the NAEP Advanced level, the proportion 
of White fourth-graders to Black or Hispanic fourth-graders was about 3:1. 


Among students who were eligible for the NSLP, 50 percent performed below NAEP Basic, 
compared to 19 percent of non-eligible students. At the NAEP Advanced level, only 2 percent of 
NSLP-eligible students performed at this level while 17 percent of students who were not NSLP- 
eligible performed at this level. 
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Table 2. Percentage of fourth-graders by NAEP achievement level and selected student 
characteristics: 2018 


below NAEP NAEP NAEP 
Student characteristics NAEP Basic Basic — — see 


All students 
Race/ethnicity 
White 27 30 al 12 100 
Black 51 30 15 4 100 
Hispanic 46 a2 18 4 100 
NSLP eligibility 
Eligible 50 a2 16 2 100 
Not eligible 19 28 36 17 100 


NOTE: Results are not reported for Asian students and students of other races/ethnicities (including American Indian/Alaska Native, 
Pacific Islander/Native Hawaiian, and Two or more races) because sample sizes were too small to meet the NAEP reporting standard for 
robust estimation. Black includes African American, and Hispanic includes Latino. Race categories exclude Hispanic origin. For National 
School Lunch Program (NSLP) eligibility, about 2 percent of students lacked valid eligibility information. These students were also 
excluded because of small sample size. Learn more about the NAEP achievement levels here. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


Percent distribution by below NAEP Basic subgroup and selected student 
characteristics 


We then dug deeper into the distribution of racial/ethnic groups among students performing 
below NAEP Basic. To examine this group more closely, students performing below NAEP Basic 
were evenly divided into three groups based on the NAEP reading score distribution.’ The three 
groups were labeled below NAEP Basic Low (i.e., the bottom one-third of students performing 
below NAEP Basic), below NAEP Basic Medium (i.e., the middle one-third of students performing 
below NAEP Basic), and below NAEP Basic High (i.e., the top one-third of students performing 
below NAEP Basic) to indicate the reading skill variation within the below NAEP Basic subgroup. 
Students’ oral reading performance was then compared across these subgroups. 


As shown in table 3, Black students were also overrepresented in the lowest below NAEP Basic 
subgroup—below NAEP Basic Low. While 26 percent of the below NAEP Basic White students 
performed at the below NAEP Basic Low level, 40 percent of the below NAEP Basic Black fourth- 
graders fell into this group. Because 51 percent of Black students were in the below NAEP Basic 
group, it means that 20 percent (or 1 out of every 5 Black fourth-graders) were at the lowest 
end of the below NAEP Basic level (51 percent x 40 percent = 20 percent). Similarly, 37 percent 
of Hispanic fourth-grade students who performed below NAEP Basic were in the below NAEP 
Basic Low subgroup. The opposite pattern is observed for the below NAEP Basic High group (i.e., 
39 percent White, 28 percent Black, and 30 percent Hispanic). 


° See appendix A for details about how below NAEP Basic subgroups are constructed. 
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Regarding NSLP eligibility, about 41 percent of non-eligible students performed at the below 
NAEP Basic High level. NSLP-eligible students were nearly equally divided among the three below 
NAEP Basic subgroups. About 35 percent of the NSLP-eligible students performed at the below 
NAEP Basic Low level (table 3). 


Table 3. Percentage of fourth-graders performing below NAEP Basic by below NAEP Basic 
subgroup and selected student characteristics: 2018 


below 
NAEP Basic 
Student characteristics Low 
All students 33 33 33 100 
Race/ethnicity 
White 26 35 39 100 
Black 40 31 28 100 
Hispanic 37 33 30 100 
NSLP eligibility 
Eligible 35 34 31 100 
Not eligible oF a2 41 100 


NOTE: Rows may not sum to totals because of rounding. Refer to Table 2 for the percentage of fourth-graders performing below NAEP 
Basic. Results are not reported for Asian students and students of other races/ethnicities (including American Indian/Alaska Native, 
Pacific Islander/Native Hawaiian, and Two or more races) because sample sizes were too small to meet the NAEP reporting standard for 
robust estimation. Black includes African American, and Hispanic includes Latino. Race categories exclude Hispanic origin. For National 
School Lunch Program (NSLP) eligibility, about 2 percent of the students lacked valid eligibility information. These students were also 
excluded because of small sample size. Learn more about the NAEP achievement levels here. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


Passage reading WCPM by NAEP achievement level and selected student 
characteristics 


Here, we report the average oral reading performance by student group and NAEP achievement 
level to examine whether the relationship between students’ oral reading skills and NAEP reading 
performance varied across student groups. In this section, only results by race/ethnicity for 
passage reading WCPM are reported. Results of other key student groups and the other five ORF 
measures are reported in appendix tables C-9 to C-14. 


As noted earlier (see table 1), on average, students read 120 words correctly in one minute— 
about the level of those who performed at the NAEP Basic level. Students performing at the 
NAEP Proficient and NAEP Advanced levels read about 142 words and 160 words correctly in 
one minute, respectively (see table 4). In contrast, students who performed below NAEP Basic 
on average read 91 words correctly in one minute, significantly lower than those students who 
performed at the NAEP Basic level (123 WCPM). 


The 2018 NAEP Oral Reading Fluency Study|Results 19 


Some of these results were similar to the national ORF percentile norms for the winter 
administration!’ (Hasbrouck and Tindal 2017), which are often used in schools and districts as a 
threshold to determine students who need additional instructional support. The average passage 
reading WCPM scores for students performing at the NAEP Basic (123) and NAEP Proficient (142) 
levels were similar to the scores of students performing at the 50th (120) and 75th (143) percentiles. 
However, the average passage reading WCPM for students performing below NAEP Basic (91) fell 
below the 25th percentile (96) as did the average passage reading WCPM for students performing 
at NAEP Advanced (160) compared to the 90th percentile (168) of the national norm. 


When students’ oral reading skills were compared by student subgroup, the results indicated 
that Black and Hispanic students read passages at a lower rate and accuracy than White students 
across the entire distribution. Importantly, at the below NAEP Basic level, the average WCPM 
for Black students (84) was lower than the average WCPM for White students (96), as displayed 
in table 4. Furthermore, the pattern of slowdown was consistent across NAEP levels within each 
demographic group. For other ORF measures by NAEP achievement level, see appendix table C-7. 


Table 4. Average passage reading words correct per minute by NAEP reading achievement 
level and selected student characteristics: 2018 


below NAEP NAEP NAEP 
Student characteristics NAEP Basic Basic ae ae srr 


All students 
Race/ethnicity 
White 96 125 145 162 100 
Black 84 122 133 146 100 
Hispanic 90 120 137 152 100 
NSLP eligibility 
Eligible 90 121 137 148 100 
Not eligible 95 126 145 161 100 


NOTE: Results are not reported for Asian students and students of other races/ethnicities (including American Indian/Alaska Native, 
Pacific Islander/Native Hawaiian, and Two or more races) because sample sizes were too small to meet the NAEP reporting standard for 
robust estimation. Black includes African American, and Hispanic includes Latino. Race categories exclude Hispanic origin. For National 
School Lunch Program (NSLP) eligibility, about 2 percent of the students lacked valid eligibility information. These students were also 
excluded because of small sample size. Learn more about the NAEP achievement levels. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


© The researchers compiled a set of national ORF norms for grades 1-6 that identified performance benchmarks at the beginning (fall), 
middle (winter), and end (spring) of the year. An individual student’s WCPM score can be compared to these benchmarks and determined 
to be either significantly above the benchmark, at the expected benchmark, below the benchmark, or significantly below the benchmark. 
The winter ORF norm is reported in this study because the NAEP 2018 ORF study was conducted during the first quarter of 2018 and they 
both measured the same construct (i.e., passage reading WCPM). 
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Oral Reading Fluency Performance and Foundational Skills 
by below NAEP Basic Subgroups 


The following figures (figures 9 to 14) show that, moving from the highest achievement level 
(NAEP Advanced) down to NAEP Basic and then through the three below NAEP Basic subgroups, 
there was a pattern of steadily decreasing performance on each measure of oral reading fluency, 
word reading, and phonological decoding. The means for passage reading WCPM, passage reading 
rate, passage reading accuracy, passage reading expression, and pseudoword reading WCPM 
were statistically different for each adjacent group." For word reading WCPM, all adjacent group 
comparisons were significant except for the difference between NAEP Advanced and NAEP 
Proficient (figure 13). 


As figure 9 shows, the difference in passage reading WCPM between below NAEP Basic High and 
below NAEP Basic Low was 38,’ which was similar to the difference between NAEP Advanced 
and NAEP Basic (37 WCPM). The average passage reading WCPM across all levels was 120. To 
help put this number in perspective, based on the 2003 National Assessment of Adult Literacy 
(NAAL) ORF data, a skilled adult reader reads orally at 166 to 178 words correctly per minute (Baer 
et al. 2009). This indicates that there is room for improvement even for fourth-grade students 
performing at the NAEP Proficient level (142 WCPM) and considerable room for improvement 
for fourth-grade students performing at the NAEP Basic level (123 WCPM). 


1 All comparisons were conducted with an alpha level of 0.05, with multiple pairwise comparison adjustments applied when needed using 
the False Discovery Rate (FDR) procedure. 
” Unrounded numbers were used for calculating the differences between the estimates. 
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Figure 9. Average passage reading WCPM by NAEP reading achievement level and 
below NAEP Basic subgroup: 2018 
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* Statistically significant difference compared to the next higher NAEP reading achievement level category, p < .05. All 
comparisons were conducted with an alpha level of 0.05, with multiple pairwise comparison adjustments applied using the 
False Discovery Rate (FDR) procedure. 

NOTE: WCPM is an abbreviation for words correct per minute. The positions of the data points in the graphics are based on 
the unrounded numbers. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Figure 10. Average passage reading rate by NAEP reading achievement level and 
below NAEP Basic subgroup: 2018 
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* Statistically significant difference compared to the next higher NAEP reading achievement level category, p < .05. All 
comparisons were conducted with an alpha level of 0.05, with multiple pairwise comparison adjustments applied using the 
False Discovery Rate (FDR) procedure. 

NOTE: The positions of the data points in the graphics are based on the unrounded numbers. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Perhaps the most noticeable difference among students performing below NAEP Basic was 
passage reading accuracy (figure 11). The below NAEP Basic Low students’ passage reading 
accuracy was 82 percent, about 9 percentage points” and 12 percentage points lower than the 
below NAEP Basic Medium and High subgroups, respectively. Eighty-two percent accuracy may 
not seem problematic, but in practical terms, it means that students misread 1 out of every 
six words. Students with such a frequent misreading of words most likely will have difficulty 
understanding the text they read because misread words tend to be content words necessary for 
text comprehension, not function words" (e.g., the, and, on). Furthermore, at 92 percent correct, 
the below NAEP Basic Medium group was missing 1 out of every 11 words, which is one word in 
nearly every sentence, since the average sentence length for the passages was about 14 words. 


Passage reading expression scores (figure 12) showed the same pattern as the passage reading 
WCPM. Performance declined steadily from NAEP Advanced to NAEP Proficient to NAEP Basic 
and continued to decline from below NAEP Basic High to below NAEP Basic Medium and below 
NAEP Basic Low. The average passage reading expression score for all fourth-grade students was 
at Level 4 on a scale of 0-5. That indicated that their oral reading expressed sentence structure 
and meaning, and that more than three-quarters of the words in the passage were read with 
appropriate expression. 


For all readers performing below NAEP Basic, the average score was in the Level 3 range. That 
meant that their oral reading expressed the meaning of words, phrases, clauses, and a few 
sentences, and that they read more than half of the words in the passage with appropriate 
expression. For the lowest below NAEP Basic subgroup, below NAEP Basic Low, the average 
expression score fell below Level 3. That indicated that these students tended to focus on local 
word groupings, which means that they often paused in the middle of a phrase. For example, 
the sentence “Hawaii is a warm place, but parts of it are cold” would be read as [Hawaii] [is a] 
[warm place], [but parts of] [it are] [cold]. 


3 Unrounded numbers were used for calculating the differences between the estimates. 

“ To hear an audio recording that illustrates the kind of words that were misread or read with difficulty, see https://nces.ed.gov/ 
nationsreportcard/studies/orf/illustrative_audio.aspx. 

15 Passage expression by a student in the below NAEP Basic Low subgroup is illustrated in an audio recording that can be found here: https:// 
nces.ed.gov/nationsreportcard/studies/orf/illustrative_audio.aspx. 
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Figure 11. Average passage reading accuracy by NAEP reading achievement level 
and below NAEP Basic subgroup: 2018 
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* Statistically significant difference compared to the next higher NAEP reading achievement level category, p < .05. All 
comparisons were conducted with an alpha level of 0.05, with multiple pairwise comparison adjustments applied using the 
False Discovery Rate (FDR) procedure. 

NOTE: Accuracy refers to the percentage of words that was read accurately. The positions of the data points in the graphics 
are based on the unrounded numbers. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Figure 12. Average passage reading expression by NAEP reading achievement 
level and below NAEP Basic subgroup: 2018 
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* Statistically significant difference compared to the next higher NAEP reading achievement level category, p < .05. All 
comparisons were conducted with an alpha level of 0.05, with multiple pairwise comparison adjustments applied using the 
False Discovery Rate (FDR) procedure. 

NOTE: The positions of the data points in the graphics are based on the unrounded numbers. Expression score 0 = Insufficient 
passage reading sample for accurate rating; 1 = Reading is word by word; less than a quarter of the words are read with 
appropriate expression; 2 = Reading focuses on local grouping; less than half of the words are read with appropriate 
expression; 3 = Reading expresses the meaning of words, phrases, clauses, and a few sentences; more than half of the words 
are read with appropriate expression; 4 = Reading expresses sentence structure and meaning; more than three-quarters 
of the words are read with appropriate expression; 5 = Passage is read as if for a listener and is expressive throughout. For 
detailed passage reading expression score description, see figure 2. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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By examining word-level reading apart from a passage, we could better understand the word- 
level processes that underlie fluency and passage comprehension. What we learned is that the 
foundational skills—word reading and phonological decoding—also varied widely within the 
below NAEP Basic subgroups (figures 13 and 14). 


In word list reading, students read high-frequency words that have known meanings to most 
students in fourth grade. Performance on this task was regarded an indicator of accumulating 
knowledge of printed words and an increasing ability to read words rapidly and automatically 
without effortful decoding. As shown in figure 13, word reading declined across the NAEP Basic 
level through all the below NAEP Basic subgroups. The sharpest decline was between the below 
NAEP Basic Medium and below NAEP Basic Low subgroups. 


In pseudoword list reading (i.e., reading lists of made-up, but pronounceable words), students read 
made-up words that required them to use phonological decoding skills, the skills that enable a 
reader to pronounce sequences of letters based on knowledge of spelling-sound correspondences 
and orthographic patterns. 


Like word reading skills, pseudoword reading skills declined across the NAEP reading achievement 
levels, including the below NAEP Basic subgroups (figure 14). The decline in mean performance 
was especially sharp between the below NAEP Basic Medium and below NAEP Basic Low subgroups. 
Moreover, there was a wide range within the below NAEP Basic subgroups. Fourth-graders in 
the below NAEP Basic High group read almost twice as many words correctly per minute (19) 
as those in the below NAEP Basic Low group (11), as shown in figure 14. The average number of 
pseudowords read correctly per minute was 22 for all fourth-grade students. 
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Figure 13. Average word reading WCPM by NAEP reading achievement level and 
below NAEP Basic subgroup: 2018 
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* Statistically significant difference compared to the next higher NAEP reading achievement level category, p < .05. All 
comparisons were conducted with an alpha level of 0.05, with multiple pairwise comparison adjustments applied using the 
False Discovery Rate (FDR) procedure. 

NOTE: WCPM is an abbreviation for words correct per minute. The positions of the data points in the graphics are based on 
the unrounded numbers. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Figure 14. Average pseudoword reading WCPM by NAEP reading achievement 
level and below NAEP Basic subgroup: 2018 
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* Statistically significant difference compared to the next higher NAEP reading achievement level category, p < .05. All 
comparisons were conducted with an alpha level of 0.05, with multiple pairwise comparison adjustments applied using the 
False Discovery Rate (FDR) procedure. 

NOTE: Pseudoword is a made-up but pronounceable word. WCPM is an abbreviation for words correct per minute. The 
positions of the data points in the graphics are based on the unrounded numbers. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Estimated Number of Fourth-Grade Students Performing 
below NAEP Basic and in the below NAEP Basic Low Subgroup 
by Race/Ethnicity 


The 2018 NAEP ORF study provided information about skills that are needed for successful oral 
reading fluency at grade 4—as well as skills that are foundational for fluent oral reading—namely, 
word reading and phonological decoding. The results of this study indicated that more than a 
third (36 percent) of the U.S. fourth-grade students in public schools were at the lowest end 
of the NAEP achievement spectrum, performing below NAEP Basic (table 2). This percentage 
translates to an estimated 1.27 million’ fourth-graders who were below NAEP Basic in American 
public schools (3.54 million x 0.36 = 1.27). Of these 1.27 million students performing below NAEP 
Basic, an estimated 0.42 million were in the below NAEP Basic Low subgroup (1.27 x 0.33 = 0.42), 
as shown in table 5. 


By race/ethnicity groups, 51 percent of all Black and 46 percent of all Hispanic fourth-graders 
performed below NAEP Basic.” Diving deeper, the results for the three subgroups of students 
scoring below NAEP Basic—Low, Medium, and High—revealed that of the 51 percent of Black 
students who were below NAEP Basic, 40 percent were in the lowest of the three subgroups. 
Similarly, of the 46 percent of Hispanic students who were below NAEP Basic, 37 percent were 
in the lowest subgroup (table 3). 


Table 5. Estimated number of fourth-grade public school students below NAEP Basic Low: 2018 


Student group Number of students 


Number of public-school 
fourth-graders represented in 


the 2018 NAEP ORF study 3.54 million 

Number of students performing below NAEP Basic by race/ethnicity 
All students 1.27 million (36 percent of all students) 
White 0.46 million (27 percent of all White students) 
Black 0.28 million (51 percent of all Black students) 
Hispanic 0.45 million (46 percent of all Hispanic students) 


See notes at end of table. 


© According to the Digest of Education Statistics (2019), the number of fourth-grade public school students is 3.7 million. According to 
the 2019 NAEP Reading Report Card, 35 percent of the fourth-grade public school students performed below NAEP Basic, representing 
1.29 million fourth-grade students. See https://www.nationsreportcard.gov/reading/nation/achievement/?grade=4. 

1’ The percentage of Black and Hispanic students in this study is 16 percent and 27 percent, respectively. See appendix table A-2. 
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Table 5. Estimated number of fourth-grade public school students below NAEP Basic Low: 
2018—continued 


Satle(sselmeneliye Number of students 


Number of students performing below NAEP Basic Low by race/ethnicity 


All students 0.42 million (33 percent of all students performing below 
NAEP Basic) 

White 121,000 (26 percent of White students in below NAEP Basic) 
1 out of 14 White students in fourth grade 

Black 115,000 (40 percent of Black students in below NAEP Basic) 


1 out of 5 Black students in fourth grade 
Hispanic 165,000 (37 percent of Hispanic students in below NAEP Basic) 
1 out of 6 Hispanic students in fourth grade 


NOTE: Detail may not sum to totals because of rounding. Only three race/ethnicity groups (White, Black, and Hispanic) are reported in 
this report because the sample size for Asian students and students of other races/ethnicities (including American Indian/Alaska Native, 
Pacific Islander/Native Hawaiian, and Two or more races) by NAEP reading achievement level did not meet the NAEP reporting standard 
for robust estimation. Black includes African American, and Hispanic includes Latino. Race categories exclude Hispanic origin. The 
percentages and numbers presented in the table are estimates based on the 2018 NAEP ORF study participants. NAEP student sampling 
weights were applied when analyzing the data to represent the population of U.S. fourth-grade public school students. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


Profile of America’s Fourth-Graders Performing below 
NAEP Basic Low 


Figure 15 is a summary of the oral reading fluency and foundational skill characteristics of 
fourth-graders performing at the below NAEP Basic Low level. In particular, reading connected 
text—sentences and passages—appears to be a major challenge for these students. They read 
correctly at about half the rate of a fourth-grader performing at the NAEP Proficient level; they 
misread about 1 out of every 6 words and they read in a mostly arrhythmic or monotone voice 
that indicates a lack of comprehension. Instead, they tend to focus on individual words or phrases. 


The other two subgroups performing below NAEP Basic—below NAEP Basic Medium and below 
NAEP Basic High—trail the NAEP Basic level averages on almost every measure of oral reading 
fluency. 


Regarding foundational skills, students in the below NAEP Basic Low group had difficulty 
recognizing whole words that they are likely to know when they are listening or speaking. 
Also, students in the below NAEP Basic Low group showed limited knowledge of spelling-sound 
correspondences, as shown by the pseudoword reading task involving made-up but pronounceable 
words. 


It is not surprising then that fourth-graders performing in the below NAEP Basic Low group do 
not meet the requirements for the NAEP Basic level, such as figuring out the main idea of a text. 
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Figure 15. Oral reading and foundational skill characteristics of fourth-graders 
performing at the below NAEP Basic Low level: 2018 


What below NAEP Basic Low fourth-graders do 


o Read connected text with difficulty—at half the WCPM of a fourth-grader performing at the NAEP 
Proficient level 

© Misread 1 out of every 6 words 

© Focus on individual words, phrases, or clauses instead of the meanings of sentences and passages 

© Read in a voice that is arrhythmic or monotone, indicating lack of text comprehension 

© Recognize with difficulty whole words they are likely to know when listening or speaking 

° Show limited knowledge of spelling-sound correspondences 


SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Discussion 


The 2018 NAEP ORF study provides important new information about the difficulties faced by 
students performing below NAEP Basic on the NAEP reading assessment, a group that represents 
a third of the students in the country. We evenly divided this below NAEP Basic group into three 
subgroups based on their NAEP scale scores distribution and closely examined oral reading 
fluency and two aspects of fluency that have been studied before: passage reading rate and 
passage reading accuracy. We added three new measures compared to the previous NAEP ORF 
studies: passage reading expression (based on expanded rubric), word list reading WCPM, and 
pseudoword list reading WCPM. 


What we learned is that the vast majority of these skills declined at each level down the NAEP 
reading achievement levels from NAEP Advanced to NAEP Proficient to NAEP Basic, and then 
again for each level down through the three below NAEP Basic subgroups. 


Passage Reading and Performance Levels 


Passage reading: Words correct per minute (WCPM) 


As figure 9 shows, passage reading WCPM on the NAEP reading assessment was lower for students 
performing at the NAEP Proficient level than at the NAEP Advanced level and lower for students 
performing at the NAEP Basic level than at the NAEP Proficient level. These findings replicated 
the results of the 2002 NAEP Oral Reading Fluency study (Daane et al. 2005). 


In this study, we see that passage reading WCPM also declined in moving from the below NAEP 
Basic High subgroup to the below NAEP Basic Medium subgroup and then to the below NAEP 
Basic Low subgroup, so there is a lot of variation within the below NAEP Basic subgroup. It is 
striking that the difference between the lowest below NAEP Basic subgroup (71 WCPM) and the 
highest below NAEP Basic subgroup (108 WCPM) was just as large, 37 WCPM, as the difference 
between students performing at NAEP Basic and NAEP Advanced levels. 


Passage reading: Rate and accuracy 


Passage reading rate (words per minute) and passage reading accuracy (percent of words read 
correctly) were also related to NAEP reading assessment performance, but they showed somewhat 
different patterns of decline. Like passage reading WCPM, passage reading rate declined through 
the NAEP Advanced, NAEP Proficient, and NAEP Basic groups and then through the three below 
NAEP Basic subgroups (figure 10). 


For passage reading accuracy, as shown in figure 11, the NAEP Basic, Proficient, and Advanced 
groups had average passage accuracy of 96, 97, and 98 percent, respectively, which, although 
similar, were statistically different. We call attention especially to the finding of 82, 92, and 
94 percent correct, respectively, for the below NAEP Basic subgroups. These may not seem like 
large or important differences. But in practical terms, the difference between 95 percent correct 
(needed for comprehension) and 90 percent (typical instructional material) is the difference 
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between missing one word out of every 20 (about one word in every two sentences) versus missing 
one word out of every 10 (or about one word in every sentence). Because the words missed 
are likely to be content- or meaning-laden words that are important for comprehension, not 
function words (e.g., the), missing them may have a large impact on comprehension performance. 
Missing words disrupts reading rate and expressivity, which are indicators of language and 
comprehension processing. 


Passage reading: Expression 


Finally, passage reading expression scores (figure 12) showed the same pattern as the passage 
reading WCPM and passage reading rate. Performance declined steadily from NAEP Advanced to 
NAEP Proficient to NAEP Basic and continued to decline from below NAEP Basic High to below 
NAEP Basic Medium and to below NAEP Basic Low. As noted earlier, the average passage reading 
expression scale score was at Level 4, indicating that, overall, fourth-graders could “correctly 
express text and sentence structure and meaning” of continuous text. 


However, the average score was at the Level 3 range for students performing below NAEP 
Basic—meaning that these students could “correctly express the meaning of phrases and clauses 
but not the passage as a whole.” In fact, for the lowest below NAEP Basic subgroup, the average 
expression score fell in the Level 2 range—suggesting that these fourth-graders tended to focus 
on individual words, not phrases, sentences, or the passage as a whole. Thus, as reading accuracy 
and reading rate decreased, evidence of appropriate expression likewise decreased. 


Overall, there was a strong and consistent relationship between the NAEP reading assessment 
performance and passage reading WCPM, passage reading rate, passage reading accuracy, and 
expressivity. Fourth-graders performing at the NAEP Proficient and NAEP Advanced levels 
(a) read grade-level texts with accuracy (97 percent), (b) read at a rate that resulted in oral 
reading fluency scores of more than 140 WCPM, and (c) read with good expressivity. In contrast, 
students in the below NAEP Basic Low, Medium, and High groups (a) read grade-level texts with 
82-94 percent accuracy, (b) read at a rate that resulted in oral reading fluency scores of less than 
110 WCPM, and (c) read with poor expressivity. There were also marked differences among the 
below NAEP Basic subgroups. 


Word-Level Reading, Performance Levels, and Race/Ethnicity 


Unlike previous studies, this study looked at word and pseudoword reading in lists. By examining 
word and pseudoword reading apart from a passage, we could better understand the word-level 
processes that underlie passage reading, oral reading fluency, and passage comprehension. 
In word list reading, students read high-frequency words that had known meanings to most 
students in fourth grade. Performance on this task is an indicator of accumulating knowledge of 
printed words and an increasing ability to read words rapidly and automatically without effortful 
decoding. In pseudoword (i.e., made-up, but pronounceable words) list reading, students read 
novel words that required them to use phonological decoding skills, the skills that enable a reader 
to pronounce a sequence of letters based on knowledge of spelling-sound correspondences. 
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Both the word and pseudoword lists showed declines across the NAEP reading achievement 
levels, including the below NAEP Basic subgroups (figures 13 and 14). The declines in mean 
performance were especially sharp at below NAEP Basic Low. Moreover, there was a wide range 
within the below NAEP Basic subgroups. For example, fourth-graders in the below NAEP Basic 
High group read almost twice as many pseudowords correctly per minute (19) as those in the 
below NAEP Basic Low group (11). 


In short, this study showed that fourth-grade students who performed below the NAEP Basic level 
on the NAEP reading assessment—that is, over a third of all fourth-graders—lacked the fluency 
and foundational skills necessary to support continued progress in reading comprehension 
across their school careers (Wang et al. 2019). Among the analyzed racial/ethnic groups, these 
performance characteristics were particularly pronounced among Black and Hispanic students. 
Inadequate development of fluency, as revealed in slow and effortful reading, is associated with 
less enjoyment, confidence, and motivation to read, slower vocabulary growth, less learning 
from text, and less application of the more complex reading strategies that are necessary for 
learning in the content areas and academic disciplines (Mol and Bus 2011; Stanovich 1986; van 
Bergen, Vasalampi, and Torppa 2020). 
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Limitations 


First, this report is descriptive and correlational in nature. Readers should not draw causal 
inferences based on the results presented here. It is important to note that while the variables 
examined in this report are related to one another, complex interactions and relations have not 
been explored here. 


Second, our study does not include a separate measure (or measures) of language comprehension. 
That would require, for example, an orally administered vocabulary test in which students hear 
the target words without having to read them, or a listening comprehension test that taps, 
among other things, students’ knowledge of sentence structure. Despite this omission, we readily 
acknowledge that language comprehension plays a critical role in reading comprehension. Both 
language comprehension and word reading are necessary for reading comprehension, where 
word reading is defined as fast and accurate word reading (Foorman, Petscher, and Herrera 2018; 
Hoover and Gough 1990). 


Third, the content coverage of the NAEP reading assessment in the 2018 NAEP ORF study is 
smaller than the NAEP operational assessment because the 2018 NAEP ORF study did not use two 
of the 10 reading blocks from that were released in 2017. Therefore, readers should not compare 
the NAEP reading assessment scores from this study to the reading scores from the operational 
NAEP reading assessments administered to fourth-grade students every 2 years. 
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Conclusion and Implications 


Conclusion 


The 2018 ORF study reveals that for an estimated 1.27 million’* fourth-grade public school 
students performing below NAEP Basic, and particularly for an estimated 0.42 million” fourth- 
grade students in the below NAEP Basic Low subgroup, fluent reading of connected text— 
sufficiently fast and accurate reading of sentences and passages—can be a major challenge. The 
study also shows that word reading and phonological decoding skills are underdeveloped in 
students performing below NAEP Basic, particularly for students in the below NAEP Basic Low 
subgroup. 


Students in the below NAEP Basic Low subgroup not only have difficulty reading the words in 
the text quickly and accurately but also show a lack of appropriate expression in reading out 
loud, which is an indicator of poor comprehension. This makes it difficult for them to engage 
in the cognitive processes described in the 2017 NAEP reading framework. For an illustrative 
audio recording, see https://nces.ed.gov/nationsreportcard/studies/orf/illustrative_audio.aspx. 


Implications 


The NAEP reading framework and future assessments 


First, the current reading framework does not describe any specific reading behaviors that 
characterize fourth-grade students performing below NAEP Basic. It states only that “These 
students are not necessarily nonreaders; many can complete some tasks on the assessment but 
are not able to attain the minimum score required for Basic” (National Assessment Governing 
Board 2017, p. 44). Based on the findings of this study, the new framework should incorporate 
a description of readers performing below NAEP Basic. It should acknowledge the fact that, 
compared to students performing at the NAEP Basic level or higher, students performing below 
NAEP Basic are more likely to have underdeveloped fluency, word reading, and phonological 
decoding skills. There should also be additional testing of fourth-grade students’ oral reading 
fluency and foundational skills with a subsample of the students who take the main NAEP reading 
assessment. Such testing would provide much-needed information about the students who are 
performing below NAEP Basic. 


Second, the framework (p. 4) notes that text comprehension is influenced by phonics knowledge 
and fluency; and, importantly, it recognizes that “without these foundational skills, comprehension 
will not occur.” It goes on to state a goal or aspiration for fourth-grade students that is universally 
accepted by reading experts and reading educators: “By grade 4, when the NAEP Reading 


18 This number refers to 36 percent of 3.54 million (number of public school fourth-graders represented in the 2018 ORF study) = 1.27 million. 
’° This number refers to a third of 1.27 million fourth-grade students who performed below NAEP Basic. Recall that students performing 
below NAEP Basic were evenly divided into three groups based on the NAEP reading score distribution. 
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Assessment is first administered, students should have a well-developed understanding of how 
sounds are represented alphabetically and should have had sufficient practice in reading to 
achieve fluency with different kinds of texts” (p. 4). But what if this goal has not been met? 


In the future, the framework should acknowledge that: “Although the majority of fourth-grade 
students do not have problems with fluency, word reading, and phonological decoding, these 
skills are not adequately developed for a significant percentage of readers performing below 
NAEP Basic,” as shown by the findings of the 2018 NAEP ORF report (White et al. 2021). 


Policy and research 


First, the problems of fourth-grade students performing below NAEP Basic highlighted by this 
report call for a solution-oriented discussion among education policymakers. The discussion 
may begin with recognition of the large income-based gaps in prereading skills that exist at 
kindergarten entry (Quinn 2015; Reardon and Portilla 2016) and proceed to a fresh and intensive 
look at programs of instruction in preschools and the early elementary grades, especially 
programs that enroll large numbers of Black and Hispanic children. Second, research is needed 
to determine the extent to which elementary schools teach accurate and efficient word reading 
skills, in systematic ways that improve oral reading fluency and reading comprehension, as 
supported by a large body of reading research (e.g., Castles, Rastle, and Nation 2018). This is a 
topic that is being vigorously debated in policy circles at the present time. 
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Appendix A. Technical Notes 


This appendix provides additional details about (a) the task development framework, (b) sampling, 
(c) weighting and variance estimation, (d) statistical testing procedures used to analyze the data 
for the 2018 NAEP ORF study, (e) the scoring procedures for the ORF data, and (f) the procedures 
used to construct the below NAEP Basic subgroups. 


Task Development Framework 


The word and pseudoword lists described below were a subset of a larger list developed for use 
in the Fluency Addition to the National Assessment of Adult Literacy (FAN). They were designed 
for adults with very low reading skills and were not intended to address the full range of adult 
abilities. Our own cognitive laboratory studies confirmed that these words were within fourth 
graders’ ability. These word lists were developed by Dr. Richard Venezky, based on principles 
derived from clinically-valid measures of children’s acquisition of phonological decoding and word 
recognition—specifically measures such as the Woodcock Johnson Word Attack and Letter-Word 
Identification (Woodcock, Mather, and Schrank 2004) and the Tests of Word Reading Efficiency 
(Torgesen, Wagner, and Rashotte 1999). 


Framework for word lists 


Three classes of real words were selected from the Kucera and Francis (1967) corpus: (1) 2- to 
5-letter, single-syllable words, (2) 2-syllable words, and (3) 3- to 5-syllable words. These words 
were then combined to form three lists. The goal was to construct lists in which the structure of 
words became progressively more complex, while maintaining a relatively high word frequency 
and familiarity to a general population. The number of letters and syllables were the primary 
indices of complexity, as they have been repeatedly shown to be the chief indicators of word 
naming accuracy and rates. In forming parallel lists, an attempt was made to separate any words 
that appeared closely related with respect to phonology, orthography, or semantics and therefore 
might cause confusion for respondents or scorers (it/at, then/than, yes/no, more/most). The 
highest frequency words, hence those most likely to be familiar to fourth-grade students, were 
chosen from the three lists for use in this study. 


Framework for pseudoword lists 


Three classes of pseudowords were developed, with subclasses within each: (1) simple, invariant; 
(2) simple, variant; and (3) multisyllabic. In each of these classes, a variety of pseudowords were 
constructed to test decoding ability. All of the pseudowords follow strict structural rules for 
English words. For example, no single-vowel item ends in a single s, l, or f, since these letters 
usually double in such positions (e.g., class, call, off). That is, the class of consonant-vowel- 
consonant (CVCs) with a final s, f, or | is restricted to a small group of mostly function words 
and shortened forms: of, al, is, as, el, us. Less frequently occurring letters (e.g., j, x, z) are used 
sparingly and no items have pronunciations that sound like common English words. (It is nearly 
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impossible to totally avoid pseudowords that sound like rarer dictionary entries.) The simplest 
sound and spelling patterns, hence most decodable by fourth-grade students, were chosen from 
the three lists for use in this study. 


Sampling 


The 2018 NAEP ORF study includes fourth-graders attending public schools. About 2,000 students 
from 220 schools from across the country were sampled. Public schools with over 75 percent of 
students eligible for the National School Lunch Program (NSLP) were sampled at twice the rate 
of the chance of selection of school of the same size with a lower percentage of NSLP eligible 
students. The sample was weighted to represent the total population of U.S. public school fourth- 
graders. Table A-1 presents the school and student response rates. 


Table A-1. NAEP Oral Reading Fluency study participant sample size and response rate: 2018 


Number of samples Weighted response rate 
REriihe) (3 participated (percent) 
School 180 85 


Student 1,800 88 


NOTE: The number of schools is rounded to the nearest ten, and the number of students is rounded to the nearest hundred. 
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


Among the sampled students, about 60 students were excluded from the NAEP reading assessment 
and the NAEP ORF assessment because they took an alternate assessment or NAEP did not allow 
the accommodations they requested. An additional 100 students were excluded from the sample 
used for the analysis because their audio responses were not captured properly due to technical 
failure. None of the other students were excluded from the oral reading assessment because of 
difficulty completing the oral reading tasks presented to them. 


The statistical standards of the National Center for Education Statistic (NCES) require a 
nonresponse bias analysis to be completed if the weighted response rate for a sample is less 
than 85 percent. Because both the weighted school and student response rates are above this 
threshold, a nonresponse bias analysis was not conducted. 


Demographic characteristics of the 2018 NAEP ORF sample 


Table A-2 displays the percentage distribution of the key subgroups included in the 2018 NAEP 
ORF study. 


1 Students who take alternate assessments based on alternate achievement standards for states’ high stakes achievement tests were 
excluded from the 2018 NAEP ORF study following the NAEP exclusion policy. For details, see https://www.nagb.gov/content/nagb/assets/ 
documents/policies/naep_testandreport_studentswithdisabilities pdf. 
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Table A-2. Weighted percentage distribution of key subgroups: 2018 NAEP Oral Reading 


Fluency study sample 
Gender 
Male 51 (0.7) 
Female 49 (0.7) 
Race/ethnicity 
White 48 (1.9) 
Black 16 (1.2) 
Hispanic 27 (1.7) 
Asian 4 (0.7) 
Other 5 (0.5) 
National School Lunch Program (NSLP) status 
Not eligible 44 (2.5) 
Eligible 54 (2.3) 
Information not available a (ig) 
English Language Learner (ELL) status 
Non-ELL 89 (1.1) 
ELL status 11 (1.1) 
Students with disability (SD) status 
Non-SD 88 (1.0) 
SD 12 (1.0) 


NOTE: Black includes African American, and Hispanic includes Latino. Race categories exclude Hispanic origin. Other includes American 
Indian/Alaska Native, Pacific Islander/Native Hawaiian, and Two or more races. Standard errors are in parenthesis. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


Treatment of missing data 
All six measures of oral reading had some missing data for one of the following reasons: 
© Administrative error: These errors include any technical failure with a recording device. 


O Insufficient audio responses: If there is insufficient student speech (e.g., not enough 
words read by a student), scores were not estimated because the available oral reading 
data could not yield reliable estimates. 


© Response anomaly: These cases include (a) extremely quiet reading, which prevented 
the automatic speech analysis system and human scorers from transcribing the audio 
reading; (b) possible off-task or irrelevant speech; and (c) anything unusual or unexpected 
in the recording that prevented the speech analysis system and human scorers from 
transcribing it. 
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Missing data on five out of six oral reading variables (i.e., word reading WCPM, pseudoword 
reading WCPM, passage reading WCPM, passage reading rate, and passage reading accuracy) were 
imputed using students’ demographic characteristics (i.e., gender, race/ethnicity, National School 
Lunch Program eligibility status, English language learner status, and disability status), two oral 
reading feedback questions,’ and all available observed oral reading data by the multivariate 
imputation by chained equations (MICE) method (van Buuren and Groothuis-Oudshoorn 2011). 
Imputed data were used for all analyses involving these five oral reading variables (n = 1,800). 
However, it is important to note that variance associated with multiple imputation was not 
accounted for when computing the standard errors; therefore, the standard errors obtained 
from the final analyses were likely underestimated. 


Missing data for the passage reading expression variable were not imputed; instead the missing 
data on the expression variable were excluded from the analysis (n = 20) involving passage 
reading expression. 


Scaling of NAEP Reading Assessment 


The NAEP reading assessment used for the 2018 NAEP ORF study consisted of eight reading blocks 
from the 2017 grade 4 NAEP reading assessment. There had been 10 blocks of reading items in the 
2017 NAEP assessment, but two were released to the public as part of the Nation’s Report Card. 
Therefore, only eight blocks were available for use by the ORF study in 2018. Those eight blocks 
of items were assembled into eight booklets, each of which included two blocks. To assemble the 
booklets, each block appeared once in the first position and once in the second position. Every 
student participating in the ORF study received one reading booklet of two blocks of items. 


Similar to the analyses of other operational NAEP reading assessment data, because each student 
did not answer all the reading items, a simple average percent correct would not allow reporting 
results that are comparable for all students; therefore Item Response Theory (IRT) was used to 
estimate average proficiency of various groups of students using their item responses and item 
characteristics such as difficulty and discrimination. IRT models the probability of answering a 
question correctly as a mathematical function of proficiency or skill. The main purpose of IRT 
analysis is to provide a common scale on which the performance of students receiving different 
booklets of items can be placed. 


During the NAEP scaling process, item parameters (e.g., difficulty and discrimination) are 
typically estimated for each item based on student response data and item type. Given the small 
sample size of the ORF study (about 500 students per item) and the fact that the same blocks 
of items from 2017 were administered, item parameters were not estimated for the 2018 ORF 
study. Instead, the item parameter estimates of the 2017 NAEP grade 4 reading assessment were 
directly applied to the ORF data. 


To estimate the distributions of the reading scale scores, population-structure models were 
used along with IRT models to appropriately estimate population and subgroup distributions of 


? Two feedback questions used for the imputation are: (a) How often have you read out loud in school or at home or anywhere in this school 
year? and (b) How difficult was this reading-out-loud test? 
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reading scale scores. The group-defining variables used in the ORF population-structure model 
included major demographic variables (i.e., gender, race/ethnicity, NSLP status, ELL status, and 
IEP status) and all ORF variables as main effects, as well as the one-way interactions between 
demographic variables and the ORF variables. 


Because the 2017 grade 4 NAEP reading item parameter estimates were used in the scaling of the 
ORF NAEP reading assessment data, the same transformation constants used in the 2017 NAEP 
reading assessment were then applied to the ORF data, thus linking the ORF reading assessment 
results to the previously established NAEP grade 4 reading scale. In a nutshell, ORF reading data 
analyses largely followed the NAEP data analysis procedure. For details about NAEP analyses and 
scaling, refer to NAEP Technical Documentation on the Web. 


Weighting and Variance Estimation 


A complex sample design was used to select the students to participate in the 2018 NAEP ORF 
study. The properties of a sample selected through a complex sample design could be different 
from a sample that is chosen randomly, in which every student in the target population has an 
equal chance of selection and in which the observations from different sampled students can be 
considered to be statistically independent of one another. Therefore, to account for the fact that 
the probabilities of selection were not identical for all students using this data collection design, 
sampling weights were used during the analysis of the assessment data. These weights included 
adjustments for school and student nonresponse. Use of such sampling weights yielded appropriate 
estimates of population characteristics. In addition to obtaining population-appropriate estimates, 
two components of uncertainty associated with these estimates were obtained. First, sampling 
errors of estimates were calculated using a set of 62 replicate weights and the jackknife repeated 
replication method to account for the degree of uncertainty associated with the sample. Second, 
the degree of imprecision associated with the reading assessment measure was also taken into 
consideration by using a set of 20 plausible values drawn from a conditional ability distribution 
for each student. Using student final sampling weights, a set of 62 replicate weights, and a set 
of 20 plausible values yielded results with an accurate population estimate and standard error 
that takes into account the sampling and measurement error. 


All reported average scores and percentages are estimates and subject to a level of uncertainty. 
The degree of uncertainty is reflected in the standard errors for each of the estimates included 
in the report, which accounts for the sampling errors and measurement errors described above 
(see appendix C for standard errors for the reported estimates). 


Statistical Testing Procedures 


For all statistical comparisons, statistical significance was determined at the alpha level of .05; 
that is, there is no more than a5 percent chance that differences could be attributed to chance. 
Significant results were calculated using the False Discovery Rate (FDR) procedure, a standard 
multiple-comparison procedure used for NAEP, based on the number of significance tests made. 
For details about the FDR procedure, refer to NAEP Technical Documentation on the Web. 
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To compare oral reading performance among key subgroups (see figures 3-7) and by NAEP 
achievement levels (see figures 9-14), a two-tailed independent sample t test was conducted. 
The formula used to compute the t statistic is follows: 


P1—P2 
t= 2 2 
J (SE,” + SE”) 


where P1 and P2 are the average oral reading scores (e.g., average passage WCPM scores for 
male and female or average passage WCPM for students performing at NAEP Basic and NAEP 
Proficient) to be compared and SE, and SE, are their corresponding standard errors. Note that 
for the NAEP achievement level comparisons, only adjacent groups were compared instead of 
making all possible comparisons. 


Scoring of Passage Variables 


The 2018 NAEP ORF study used a new automatic speech analysis system that transcribed students’ 
oral reading and calculated their accuracy, rate, and words correct per minute (WCPM) by aligning 
the transcription to the passage text. That is, the automatic speech analysis system transcribed 
from the first to the last word what a student read aloud, then it calculated the number of words 
read by the student (i.e., span length). It also calculated the duration of students’ oral reading 
relevant to the passage (i.e., span duration). Lastly, the system counted the number of words 
that were correctly read in the correct order within the span length (i.e., number of correctly 
read words in text span). These three pieces of information were used to calculate passage rate, 
accuracy, and words correct per minute (WCPM). 


For example, a student attempted to read 90 words (span length) from the Guided Dogs passage 
that is 152 words long (see figure B-1), which took the student 49 seconds (span duration). Among 
the 90 words, 86 words were read correctly in the correct order (number of correctly read words 
in text span). Given this, 


O. Passage reading rate (words per minute): 90 (span length) / 49 (span duration in seconds) 
X 60 (seconds) = 110 words per minute; 


O Passage accuracy (percent of words read correctly): 86 (number of correctly read words 
in text span) / 90 (span length) x 100 = 96 percent accuracy; 


© Passage WCPM: 86 (number of correctly read words in text span) / 49 (span duration in 
seconds) x 60 (seconds) = 105 words correctly read per minute. 
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Table A-3. Interrater reliability for passage scoring: 2018 


Maximum number of Number of Interrater 
Passage words audio recordings reliabilit 
162 279 99 


Passage 1 

Passage 2 153 275 98 
Passage 3 162 283 oo 
Passage 4 152 283 94 


NOTE: Passage interrater reliability is a correlation between the counts of words correctly read using the machine transcriptions and 
those using the human transcriptions of the same audio recording. The final passage interrater reliability (.96) is the average correlation 
across the four passages. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


Constructing below NAEP Basic Subgroups 


To examine the relationship between oral reading fluency and reading comprehension skills with 
an emphasis on the students performing below NAEP Basic (see the Average ORF Performance 
by NAEP Reading Achievement Levels section), these students were divided into three quantile 
groups based on students’ scores from the NAEP reading assessment (i.e., each group represents 
one-third of the students performing below NAEP Basic). As seen in table A-4, each of the below 
NAEP Basic subgroups includes 12 percent of the grade 4 public school students. The average 
NAEP reading score for each of the three below NAEP Basic subgroups ranged from 155 to 202. 


Table A-4. Interrater reliability for passage scoring: 2018 


Unweighted Weighted 
percentage percentage Weighted 

Achievement level (percent) (percent) 
below NAEP Basic Low 14 12 (1.0) 422,300 155 (3.3) 
below NAEP Basic Medium 13 12 (0.9) 424,000 184 (2.2) 
below NAEP Basic High Hie) 12 (0.9) 422,700 202 (1.9) 
NAEP Basic 30 30 (1.6) 1,070,000 223. (1.2) 
NAEP Proficient 23 25 (1.6) 880,600 249 (1.4) 
NAEP Advanced 8 9 (1.1) 323,700 273 (2.9) 

Total 100 100 = (ft) 3,543,300 219 (1.4) 
t Not applicable. 
NOTE: Standard errors are in parentheses. Detail may not sum to totals because of rounding. The weighted sample size is rounded to the 
nearest hundred. 


SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Appendix B: Sample Oral Reading 
Passage 


This appendix presents a sample oral reading passage used in the NAEP 2018 Oral Reading 
Fluency study. This is one of the four reading passages administered to students in the study. 
This passage, as well as the other three passages, represents the complexity level of the texts 
that fourth-graders typically read, according to several readability measures we used, including 
the Flesch-Kincaid Grade Level measure (Kincaid et al. 1975). 


The average scores for this passage, including words correct per minute, rate, accuracy and 
expression, are provided in table B-1 below. More information about the sample reading passage, 
such as sample digital recordings of student oral reading responses, can be found on the NAEP 
ORF website at https://nces.ed.gov/nationsreportcard/studies/orf/illustrative_audio.aspx. 


Figure B-1. Sample oral reading passage: 2018 


Guide Dogs (152 words) 


Guide dogs lead very interesting lives. For 10 or 12 years, they are in charge of guiding 
a blind person. To do this job they must be smart, gentle, and very well trained 


Most guide dogs are born at a kennel. Since dogs are gentler when raised by a family, 
the dogs are given to families soon after they are born. When the dogs are about 
14 months old, they return to the kennel to be trained. 


The dogs train in large groups for about three months, and they know more at the end 
of that time than most dogs will ever learn. But the training isn’t over yet. Their new 
masters arrive and they train together as a team for one more month. 


At the end of that time they are ready for the world. Guide dogs form strong bonds 
with their masters, and they keep them company all the time. 


SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


Fourth grade children are expected to read passages that introduce them to a topic they are 
unfamiliar with and/or passages that build on and extend existing knowledge. In the case of the 
Guide Dogs passage, most children already know something about dogs, and they understand 
the general meaning of the word guide. In reading the passage, they learn that there are dogs 
trained specifically to help blind people called guide dogs, even if this concept has not been part 
of their day to day experience. Note that the second sentence essentially provides the meaning 
of guide dog: “For 10 or 12 years, they are in charge of guiding a blind person.” 
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Fourth grade children are also expected to use their phonological decoding skills and context to 
infer the meaning of words that they have not encountered in print and perhaps not even heard. 
In the case of the word “kennel,” for example, the word is regular and decodable. The passage 
has two sentences: “Most dogs are born at a kennel.” And “When the dogs are about 14 months 
old, they return to the kennel to be trained.” These sentences provide enough context to infer 
what a kennel is. The inferred word meaning may not be as deep or as precise as the child could 
get from looking up the word in a dictionary, but enough to make sense of the passage. 


Table B-1. Average oral reading performance for the “Guide Dogs” passage: 2018 


IW CER gS Average 


Passage reading words correct per minute (WCPM) 113. (1.2) 
Passage reading rate: words per minute 119 (1.1) 
Passage reading accuracy: percentage of words read correctly 94 (0.3) 
Passage reading expression 3.8 (0.03) 


NOTE: Standard errors are in parentheses. 


SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Appendix C: Estimates and 
Standard Errors for Tables and 
Figures 


This appendix contains the standard errors for the estimated averages and percentages in all 
tables and figures throughout this report. Some additional data not discussed in detail in the 
report are also included here. 


Table C-1. Estimates and standard errors for figure 3. Average oral reading performance 
of U.S. fourth-graders by gender: 2018 


ORF measure | DYeK Yor gh ho} a Loyal Average 


Oral reading fluency Passage reading words correct per minute(WCPM) 120 (1.2) 
Passage reading rate: Words per minute 126 (1.1) 
Passage reading accuracy: Percentage of words 
read correctly 94 (0.2) 
Passage reading expression 4.0 (0.03) 
Word reading Word list reading: WCPM 51 (0,5) 
Phonological decoding Pseudoword list reading: WCPM 22 (0.3) 


NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. 
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


Table C-2. Estimates and standard errors for figure 3. Average oral reading performance 
of U.S. fourth-graders by gender: 2018 


Measure Measure description WES Female 


Oral reading fluency — Passage reading words correct per 
minute (WCPM) 118 (1.6) 122* (1.3) 


Passage reading rate: Words per minute 125 (1.4) 128 (1.3) 
Passage reading accuracy: Percentage of 


words read correctly 94 (0.3) 95* (0.3) 
Passage reading expression 3.9 (0.03)  4.0* (0.03) 
Word reading Word list reading: WCPM 51 (06) 51 (0.7) 
Phonological decoding Pseudoword list reading: WCPM 23. (0.4) 21* (0.4) 


* Statistically significant score difference when compared to male students, p < .05. 

NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Table C-3. Estimates and standard errors for figure 3. Average oral reading performance of U.S. fourth-graders by gender: 


2018 
Oral reading fluency Passage reading words 
correct per minute (WCPM) 128 (1.6) 106* (2.2) 110* (1.6) 137 (61) 123 (4.9) 
Passage reading rate: Words 
per minute 133 (1.4)| 114* (2.1)| 118* (1.4)| 144 (5.5)| 130 (4.5) 
Passage reading accuracy: 
Percentage of words read 
correctly 96 (0.3) 92* (0.6) 93* (0.4) 94 (1.3) 95 (0.9) 
Passage reading expression 4.1 (0.04) 3.7* (0.06) 3.7* (0.04) 4.2 (0.11) 4 (0.13) 
Word reading Word list reading: WCPM 53 (0.6) 47* (0.9) 50* (0.7) 52 (2.3) 49 (2.6) 
Phonological decoding = Pseudoword list reading: 
WCPM 24 (0.5) 18* (0.7) 21* (0.5) 26 (1.9) 23 (1.4) 


* Statistically significant score difference compared to White students, p < .05. All comparisons were conducted with an alpha level of 0.05 with White students as a reference group. The 
False Discovery Rate (FDR) procedure was used to adjust for multiple pairwise comparisons between White and four other racial/ethnic groups (i.e., 4 comparisons). 

NOTE: Pseudoword is a made-up but pronounceable word. Black includes African American, and Hispanic includes Latino. Race categories exclude Hispanic origin. Other includes 
American Indian/Alaska Native, Pacific Islander/Native Hawaiian, and Two or more races. Standard errors are in parentheses. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2018 Oral Reading 
Fluency study. 
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Table C-4. Estimates and standard errors for figure 5. Average oral reading performance 
of U.S. fourth-graders by National School Lunch Program (NSLP) eligibility status: 2018 


Information 


IES) Bed not 
Measure Measure descri paleyeee=) btsa le) (3 EMENI Ele) Ce 


Oral reading fluency _ Passage reading words 
correct per minute 


(WCPM) 133. (1.4) —-108* =(1.3) 141 = (4.2) 
Passage reading rate: 
Words per minute 138 §=(1.3) +116* (1.2) 145 (3.9) 


Passage reading accuracy: 
Percentage of words read 


correctly 96 (0.3) 93* (0.3) 97* (0.4) 

Passage reading 

expression 4.2 (0.03)  3.7* (0.03) 4.5* (0.04) 
Word reading Word list reading: WCPM 54 (0.7) 48* (0.7) 58* (0.7) 
Phonological decoding Pseudoword list reading: 

WCPM 25 (0.5) 19* (0.4) 29* (0.9) 


* Statistically significant score difference compared to students not eligible for NSLP, p < .05. All comparisons were conducted with an 
alpha level of 0.05 with students not eligible for NSLP as a reference group. The False Discovery Rate (FDR) procedure was used to adjust 
for multiple pairwise comparisons between students not eligible NSLP and two other NSLP status groups (i.e., 2 comparisons). 

NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


Table C-5. Estimates and standard errors for figure 6. Average oral reading performance 
of U.S. fourth-graders by English Language Learner (ELL) status: 2018 


Non-ELL 


Oral reading fluency Passage reading words correct 
per minute (WCPM) 123 (1.2) 99* (2.6) 


Passage reading rate: Words per minute 128 (1.1)  109* (2.3) 
Passage reading accuracy: Percentage of 


words read correctly 95 (0.2) 90* (0.8) 
Passage reading expression 4.0 (0.03) 3.5* (0.07) 
Word reading Word list reading: WCPM 52. (0:5) 46* (1.2) 
Phonological decoding Pseudoword list reading: WCPM 23 (0.3) 18* (0.6) 


* Statistically significant score difference when compared to non-ELL students, p < .05. 

NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. The results for English language learners 
are based on students who were assessed and cannot be generalized to the total population of such students. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Table C-6. Estimates and standard errors for figure 7. Average oral reading performance 
of U.S. fourth-graders by students with disabilities status: 2018 


Without With 
Measure Measure description disabilities| disabilities 


Oral reading fluency Passage reading words correct 
per minute (WCPM) 124 (1.2) 92* (2.6) 


Passage reading rate: Words per minute 129 (1.1) 103* (2.3) 
Passage reading accuracy: Percentage of 


words read correctly 95 (0.2) 88* (0.8) 
Passage reading expression 4.0 (0.02)  3.3* (0.08) 
Word reading Word list reading: WCPM 53 (0.5)| —41* =~ (1.5) 
Phonological decoding Pseudoword list reading: WCPM 23 (0.3) 15* (0.9) 


* Statistically significant score difference when compared to students without disabilities, p < .05. 

NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. The results for students with disabilities 
are based on students who were assessed and cannot be generalized to the total population of such students. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 


Table C-7. Estimates and standard errors for table 2. Percentage of fourth-graders by NAEP 
achievement level and by selected student characteristics: 2018 


below NAEP NAEP NAEP 
Student characteristics NAEP Basic Basic| Proficient} | Advanced| Total 


All students (1.7) 30 (1.6) 25 (1.6) 9 (1.1) 
Race/ethnicity 
White 27 (2.2) 30 (2.6) 31 (2.4) 12 (1.9) 100 
Black 51 (4.2) 30 (3.8) 15 (2.7) 4 (1.6) 100 
Hispanic 46 (2.6) $2. (2.7) 18 (2.4) 4 (1.4) 100 
NSLP eligibility 
Eligible 50 (2.0) 32 (2.1) 16 (1.6) 2 (0.7) 100 
Not eligible 19 (2.1) 28 (2.4) 36 (2.6) 17 (2.0) 100 


NOTE: Standard errors are in parentheses. Detail may not sum to total because of rounding. Only three race/ethnicity groups (White, 
Black, and Hispanic) are reported in this report because the sample size for Asian students and students of other races/ethnicities 
(including American Indian/Alaska Native, Pacific Islander/Native Hawaiian, and Two or more races) by NAEP reading achievement 
level did not meet the NAEP reporting standard for a robust estimation. Black includes African American, and Hispanic includes Latino. 
Race categories exclude Hispanic origin. For National School Lunch Program (NSLP) eligibility, about 2 percent of the students lacked 
valid eligibility information. These students were not reported because of a small sample size. Learn more about the NAEP achievement 
levels. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Table C-8. Estimates and standard errors for table 3. Percentage of fourth-graders 
performing below NAEP Basic by below NAEP Basic subgroup and selected student 
characteristics: 2018 


below 
NAEP Basic 
RJutletsanmelatber:answtin ens Low i Total 
All students 33 (2.1) 33 (2.0) 33 (2.1) 100 
Race/ethnicity 
White 26 (4.1) 35 (4.8) 39 (5.0) 100 
Black 40 (5.1) 31 (5.5) 28 (4.9) 100 
Hispanic 37 (3.9) 33 (4.4) 30 (3.9) 100 
NSLP eligibility 
Eligible 35. (25) 34 (2.7) 31 (2.5) 100 
Not eligible 27 (5.2) 32 (6.0) 41 (6.0) 100 


NOTE: Standard errors are in parentheses. Detail may not sum to total because of rounding. Only three race/ethnicity groups (White, 
Black, and Hispanic) are reported in this report because the sample size for Asian students and students of other races/ethnicities 
(including American Indian/Alaska Native, Pacific Islander/Native Hawaiian, and Two or more races) by NAEP reading achievement 
level did not meet the NAEP reporting standard for a robust estimation. Black includes African American, and Hispanic includes Latino. 
Race categories exclude Hispanic origin. For National School Lunch Program (NSLP) eligibility, about 2 percent of the students lacked 
valid eligibility information. These students were not reported because of a small sample size. Learn more about the NAEP achievement 
levels. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Table C-9. Estimates and standard errors for table 4 and figures 9-14. Average oral reading performance by NAEP reading 
achievement level: 2018 


below NAEP NAEP NAEP 
Measure ORF variable NAEP Basic Basic| Proficient} | Advanced 


Oral reading fluency Passage reading words correct per minute 


(WCPM) 91 (1.5) 123 (1.5) 142 (16) 160 (2.4) 

Passage reading rate: Words per minute 101 (1.3) 128 (1.5) 146 (1.6) 163 (2.4) 

Passage reading accuracy: Percentage of words 

read correctly 90 (0.5) 96 (0.2) 97 (0.2) 98 (0.2) 

Passage reading expression 3.3 (0.04) 4.1 (0.03) 4.4 (0.03) 4.7 (0.04) 
Word reading Word list reading: WCPM 43 (0.9) 54 (0.6) 57 (0.8) 59 (1.2) 
Phonological decoding Pseudoword list reading: WCPM 15; (05) 23 (0.6) 27 (0.7) 31 (1.0) 


NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. All adjacent comparisons are statistically significant except for the difference in word list 
reading WCPM between NAEP Proficient and NAEP Advanced, p < .05. For instance, there was statistically significant difference in passage reading WCPM between below NAEP Basic 
and NAEP Basic, NAEP Basic and NAEP Proficient, and NAEP Proficient and NAEP Advanced. The cut scores for NAEP Basic, NAEP Proficient, and NAEP Advanced are 208, 238, and 268, 
respectively. The NAEP reading scale ranges from 0 to 500. NAEP achievement levels are performance standards that describe what students should know and be able to do. Students 
performing at or above the NAEP Proficient level on NAEP assessments demonstrate solid academic performance and competency over challenging subject matter. NAEP Proficient does 
not represent grade-level proficiency as determined by other assessment standards (e.g., state or district assessments). Learn more about the NAEP achievement levels. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2018 Oral Reading 
Fluency study. 
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Table C-10. Estimates and standard errors for average oral reading performance by NAEP achievement level and gender: 


2018 


below NAEP NAEP NAEP 
Measure ORF variable Gender | NAEP Basic Basic ome eABeE 


Oral reading fluency _ Passage reading words correct per Male 


minute (WCPM) Female 
Passage reading rate: Words per Male 
minute Female 
Passage reading accuracy: Male 
Percentage of words read correctly Female 
Passage reading expression Male 
Female 
Word reading Word list reading: WCPM Male 
Female 


Phonological decoding Pseudoword list reading: WCPM Male 
Female 


92 
101 
101 

89 

91 

3.2 
3.4 

43 

43 

16 

14 


(2.1) 
(1.8) 
(1.8) 
(1.6) 
(0.7) 
(0.6) 
(0.05) 
(0.06) 
(1.1) 
(1.2) 
(0.7) 
(0.6) 


123 
129 
128 
96 
96 
4.0 
4.1 
es) 
52 
24 
22 


(2.2) 
(2.2) 
(2.2) 
(2.3) 
(0.3) 
(0.3) 
(0.04) 
(0.05) 
(0.8) 
(1.0) 
(0.8) 
(0.8) 


143 
146 
147 
97 
97 
4,4 
4.5 
58 
56 
28 
26 


(2.2) 
(2.4) 
(2.2) 
(2.4) 
(0.2) 
(0.3) 
(0.05) 
(0.04) 
(0.7) 
(1.3) 
(0.8) 
(0.9) 


159 
164 
163 
98 
98 
4,7 
4,7 
60 


(3.3) 
(3.6) 
(3.3) 
(3.7) 
(0.2) 
(0.3) 
(0.05) 
(0.06) 
(1.3) 
(1.8) 
(1.3) 
(1.3) 


NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. WCPM is an abbreviation for words correct per minute. Learn more about the NAEP 


achievement levels. 


SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2018 Oral Reading 


Fluency study. 
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Table C-11. Estimates and standard errors for table 4: Average oral reading performance by NAEP achievement level and 
race/ethnicity: 2018 


Race/ below NAEP LV NAEP 
Measure ORF variable ethnici NAEP Basic Basic ee are 


Oral reading fluency Passage reading words correct per White 
minute (WCPM) Black 84 
Hispanic 90 
Passage reading rate: Words per _ White 105 
minute Black o5 
Hispanic 100 
Passage reading accuracy: White 91 
Percentage of words read Black 88 
correctly Hispanic 89 
Passage reading expression White 3.4 
Black 3.1 
Hispanic 3.3 
Word reading Word list reading: WCPM White 44 
Black 40 
Hispanic 44 
Phonological decoding Pseudoword list reading: WCPM _ White 16 
Black 13 
Hispanic 16 


(2.6) 
(3.2) 
(2.1) 
(2.4) 
(2.8) 
(1.9) 
(0.8) 
(1.2) 
(0.8) 
(0.07) 
(0.08) 
(0.06) 
(1.4) 
(1.5) 
(1.4) 
(1.0) 
(0.8) 
(0.7) 


122 
120 
130 
127 
125 
96 
96 
96 
4,1 
4,1 
4.0 
54 
52 
54 
23 
22 
23 


(2.2) 
(3.1) 
(2.1) 
(2.2) 
(3.1) 
(2.0) 
(0.3) 
(0.5) 
(0.4) 
(0.05) 
(0.07) 
(0.05) 
(0.9) 
(1.5) 
(1.0) 
(0.7) 
(1.4) 
(1.0) 


133 
137 
148 
138 
141 
97 
97 
97 
4.5 
4,4 
4,4 
57 
56 
58 
27 
26 
27 


(2.2) 
(3.6) 
(2.7) 
(2.1) 
(3.6) 
(2.6) 
(0.2) 
(0.7) 
(0.4) 
(0.04) 
(0.09) 
(0.07) 
(0.9) 
(1.8) 
(1.3) 
(0.9) 
(1.6) 
(1.3) 


146 
152 
165 
151 
155 
98 
97 
98 
4.8 
4.6 
4.6 
59 
58 
60 
32 
28 
30 


(2.6) 
(6.1) 
(6.9) 
(2.6) 
(6.7) 
(6.9) 
(0.2) 
(0.7) 
(0.7) 
(0.04) 
(0.15) 
(0.12) 
(1.1) 
(3.3) 
(2.3) 
(1.2) 
(2.5) 
(3.0) 


NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. WCPM is an abbreviation for words correct per minute. Only three race/ethnicity groups 
(White, Black, and Hispanic) are reported in this report because the sample size for Asian students and students of other races/ethnicities (including American Indian/Alaska Native, 
Pacific Islander/Native Hawaiian, and Two or more races) by NAEP reading achievement level did not meet the NAEP reporting standard for robust estimation. Black includes African 
American, and Hispanic includes Latino. Race categories exclude Hispanic origin. Learn more about the NAEP achievement levels. 


SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2018 Oral Reading 


Fluency study. 


Table C-12. Estimates and standard errors for table 4: Average oral reading performance by NAEP achievement level and 
National School Lunch Program (NSLP) eligibility status: 2018 


NSLP below NAEP NAEP NAEP 
Measure ORF variable status NAEP Basic Basic cea gee 


Oral reading fluency = Passage reading words correct per _ Eligible (1.6) (1.7) (2.7) (7.1) 
minute (WCPM) Not eligible 95 (3.0) 126 (2.4) 145 (1.9) 161 (2.6) 

Passage reading rate: Words per Eligible 100 (1.4) 126 (1.7) 141 (2.7) 152 (7.2) 

minute Not eligible 104 (2.9) 131 (2.5) 149 (1.9) 165 (2.6) 

Passage reading accuracy: Eligible 89 (0.6) 96 (0.2) 97 (0.3) 98 (0.6) 

Percentage of words read correctly Noteligible 91 (0.9) 96 (0.4) 97 (0.2) 98 (0.2) 

Passage reading expression Eligible 3.2 (0.05) 4.0 (0.04) 4.3 (0.05) 4.5 (0.13) 

Not eligible 3.4 (0.09) 4.1 (0.04) 4.5 (0.04) 4.8 (0.04) 

Word reading Word list reading: WCPM Eligible 43 (1.0) 53 (0.8) 55 (1.3) 56 (3.0) 
Not eligible 43 (1.7) 54 (1.1) 58 (1.0) 60 (1.3) 

Phonological decoding Pseudoword list reading: WCPM Eligible 15 (0.5) 23 (0.7) 27 (0.9) 28 (2.6) 


Not eligible 16 (1.0) 23 (0.8) 27 (0.9) 32 (1.1) 


Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. WCPM is an abbreviation for words correct per minute. About 2 percent of the students lacked 
valid information about their National School Lunch Program (NSLP) eligibility. These students are not reported by NAEP reading achievement level because the small sample size did not 
meet the NAEP reporting standard for robust estimation. Learn more about the NAEP achievement levels. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2018 Oral Reading 
Fluency study. 
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Table C-13. Estimates and standard errors for average oral reading performance by NAEP achievement level and English 
Language Learner (ELL) Status: 2018 


below NAEP NAEP NAEP 
Measure ORF variable fee NAEP Basic Basic arr Advanced 
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Oral reading fluency Passage reading words correct per Yes (2.6) (4.2) + = (t) 
minute (WCPM) No 93 (1.7) 124 (1.6) 143 160 (2.3) 
Passage reading rate: Words per Yes 98 (2.3) 123 (4.0) + (t) (7) 
minute No 102 (1.5) 129 (1.6) 147 (1.6) 163 (2.3) 
Passage reading accuracy: Yes 87. (1.1) 95 (0.8) £ (t) # = (fT) 
Percentage of words read correctly No 90 (0.5) 96 (0.2) 97 (0.2) 98 (0.2) 
Passage reading expression Yes 3.1 (0.08) 3.9 (0.09) £ (t) $= (Tt) 
No 3.3 (0.05) 4.1 (0.03) 4.5 (0.03) 4.7 (0.04) 
Word reading Word list reading: WCPM Yes 41 (1.6) 53 (1.9) - (t) = . ) 
No 43 (1.0) 54 (0.7) 57 (0.8) 59 (1.2) 
Phonological decoding Pseudoword list reading: WCPM Yes 15 (0.8) 21 (1.7) + = (t) + (ft) 
No 15 (0.6) 23 (0.6) 27 (0.7) a2. 0,0) 
t Not applicable. 


+ Reporting standards not met. 

NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. WCPM is an abbreviation for words correct per minute. Estimates for students 
performing at the NAEP Proficient and NAEP Advanced levels are not reported for ELL students because the number of ELL students in these achievement level categories do not 
meet the NAEP reporting standards for robust estimation. Learn more about the NAEP achievement levels. The results for English language learners are based on students who were 
assessed and cannot be generalized to the total population of such students. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2018 Oral 
Reading Fluency study. 


Table C-14. Estimates and standard errors for average oral reading performance by NAEP achievement level and students 
with disabilities status: 2018 


Disability below NAEP NAEP NAEP 
WERE ORF variable status NAEP Basic Basic ar ar 


Oral reading fluency = Passage reading words correct per Yes (2.7) (6.0) 
minute (WCPM) No 95 (1.7) 123 (1.5) 142 is 160 oa 
Passage reading rate: Words per Yes 92 (2.5) 127 (5.8) + (t) $+ = (Tt) 
minute No 104 (1.6) 128 (1.5) 146 (1.6) 163 (2.4) 
Passage reading accuracy: Yes 85 (1.0) 95 (1.0) = (t) + -4y) 
Percentage of words read correctly No 91 (0.5) 96 (0.2) 97. (0.2) 98 (0.2) 
Passage reading expression Yes 2.9 (0.11) 4.0 (0.11) + (t) = 7) 
No 3.4 (0.05) 4.1 (0.03) 4.4 (0.03) 4.7 (0.04) 
Word reading Word list reading: WCPM Yes 36 (1.8) 52 (2.9) = (t) i: (fF) 
No 45 (0.9) 54 (0.7) 57 (0.8) 59 (1.3) 
Phonological decoding Pseudoword list reading: WCPM Yes 12 (0.8) 20 (2.2) + s(t) t+ =§6(t) 
No 16 (0.5) 23 (0.5) 27 (0.7) 32 (1.0) 
t Not applicable. 


+ Reporting standards not met. 

NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. WCPM is an abbreviation for words correct per minute. Estimates for students performing 
at the NAEP Proficient and NAEP Advanced levels are not reported for students with disabilities because the number of students with disabilities in these achievement level categories do 
not meet the NAEP reporting standards for robust estimation. Learn more about the NAEP achievement levels. The results for students with disabilities are based on students who were 
assessed and cannot be generalized to the total population of such students. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2018 Oral Reading 
Fluency study. 
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Table C-15. Estimates and standard errors for figures 9-14. Average oral reading 
performance by below NAEP Basic subgroups: 2018 


below below below 


NAEP Basic| NAEP Basic| NAEP Basic 
Werkathas ORF variable Low i 


Oral reading fluency = Passage reading words 
correct per minute(WCPM) 71 (2.0) 95 (2.1) 108 (2.5) 
Passage reading rate: 
Words per minute 86 (2.0) 103 (2.1) 115 (2.5) 
Passage reading accuracy: 
Percentage of words read 


correctly 82 (1.1) 92 (0.7) 94 (0.5) 

Passage reading expression 2.7 (0.07) 3.4 (0.07) 3.8 (0.06) 
Word reading Word list reading: WCPM 34 (1.6) 45 (1.6) 50 (1.4) 
Phonological decoding Pseudoword list reading: 

WCPM 11 (0.7) 16 (0.9) 19 (0.8) 


NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. All adjacent comparisons are statistically 
significant except for the difference in word list reading WCPM between below NAEP Basic Medium and below NAEP Basic High, p < .05. 
For instance, there was statistically significant difference in passage reading WCPM between below NAEP Basic Low and below NAEP 
Basic Medium, and below NAEP Basic Medium and below NAEP Basic High. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP) 2018 Oral Reading Fluency study. 
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Table C-16. Estimates and standard errors for average oral reading performance by below NAEP Basic subgroups 
and gender: 2018 A 


below below below 


NUE of ode bye Ky Lol ADV ol a be Ky Com ADV ol eal Be Ky Les 
Wrkera ORF variable reryalelsyu Low i 


Oral reading fluency — Passage reading words correct per minute Male 70 (2.7) 95 (2.7) 109 (3.4) 
(WCPM) Female 71 (2.9) 94 (3.4) 107 (3.4) 

Passage reading rate: Words per minute Male 87 (2.7) 104 (2.7) 116 (3.4) 

Female 85 (3.0); 102 (3.3)| 113 (3.4) 

Passage reading accuracy: Percentage of Male 82 (1.3) 92 (0.9) 94 (0.8) 

words read correctly Female 84 (1.7) 92 (1.1) 95 (0.8) 

Passage reading expression Male 2.6 (0.09) 3.3 (0.09) 3.7 (0.08) 

Female 2.8 (0.11) 3.5 (0.10) 3.8 (0.08) 

Word reading Word list reading: WCPM Male 34 (1.7) 45 (2.2) 51 (1.8) 
Female 34 (2.4) 44 (2.0) 49 (1.9) 

Phonological decoding Pseudoword list reading: WCPM Male 12. (0.9) 7: (3) 20 (1.3) 
Female 10 (1.0) 14 (1.0) 18 (1.1) 


NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. WCPM is an abbreviation for words correct per minute. 
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2018 
Oral Reading Fluency study. 
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3 Table C-17. Estimates and standard errors for average oral reading performance by below NAEP Basic subgroups and 

z race/ethnicity: 2018 

3 below below below 

3 NAEP Basic| NAEP Basic| NAEP Basic 

@ WEENtdS ORF variable ici Low i 

Qa 

= Oral reading fluency = Passage reading words correct per minute White 73 (4.5) 99 (4.0) 110 (4.0) 

z (WCPM) Black 64 (3.0) 90 (44) 107 (49) 

iS Hispanic 73 (3.2) 94 (2.8) 106 (3.5) 

¢ Passage reading rate: Words per minute White 88 (4.5) 106 (3.8) 116 (4.2) 

= Black 80 (3.1) 98 (43) 113 (47) 

3 Hispanic 88 (2.8); 103 (3.1)| 113 (@.7) 

= Passage reading accuracy: Percentage of | White 84 (2.3) 93 (1.1) 95 (0.8) 

Q words read correctly Black 81 (2.1) 91 (13) 94 (1.1) 

E Hispanic 83 (1.8) 92 (1.1) 94 (0.9) 

= Passage reading expression White 2.8 (0.15) 3.5 (0.13) 3.8 (0.10) 

B Black 2.5 (0.12)| 3.4 (0.11)| 3.8 (0.11) 

g Hispanic 2.8 (0.13) 3.4 (0.10) 3.7 (0.09) 

e Word reading Word list reading: WCPM White 34 (3.1) 46 (2.8) 50 (2.0) 

rm Black 31 (24) 44 (2.7) 50 (2.8) 

g Hispanic 38 (2.4) 45 (2.0) 49 (2.1) 
Phonological decoding Pseudoword list reading: WCPM White 11 (1.5) 16 (1.5) 20 (1.3) 

Black 9 (0.9) 14 (1.6) 18 (1.9) 


Hispanic 13 (1.3) 17. (13) 18 (1.4) 


NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. WCPM is an abbreviation for words correct per minute. Only three race/ 
ethnicity groups (White, Black, and Hispanic) are reported in this report because the sample size for Asian students and students of other races/ethnicities (including 
American Indian/Alaska Native, Pacific Isander/Native Hawaiian, and Two or more races) by NAEP reading achievement level did not meet the NAEP reporting standard for 
robust estimation. Black includes African American, and Hispanic includes Latino. Race categories exclude Hispanic origin. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2018 
Oral Reading Fluency study. 


Table C-18. Estimates and standard errors for average oral reading performance by below NAEP Basic subgroups and 
National School Lunch Program (NSLP) eligibility status: 2018 


below below below Hd 
NSLP NAEP Basic| NAEP Basic| NAEP Basic 
Measure ORF variable status Low i 
Oral reading fluency — Passage reading words correct per minute Eligible 70 (2.2) 94 (2.4) 107 (2.9) 
(WCPM) Not eligible 72 (5.5) 94 (5.2) 110 (4.5) 
Passage reading rate: Words per minute _ Eligible 86 (2.2)| 103 (2.3); 113 (2.8) 
Not eligible 84 (5.0) 103 (5.0) 117 (4.7) 
Passage reading accuracy: Percentage of _ Eligible 82 (1.2) 92 (0.8) 94 (0.6) 
words read correctly Not eligible 84 (25) 91 (16) 94 (1.2) 
Passage reading expression Eligible 2.7 (0.08) 3.4 (0.07) 3.7 (0.07) 
Not eligible 2.7 (0.23) 3.5 (0.16) 3.8 (0.10) 
Word reading Word list reading: WCPM Eligible 35 (1.7) 45 (1.7) 49 (1.6) 
Not eligible 31 (3.9) 44 (3.6) 50 (2.5) 
Phonological decoding Pseudoword list reading: WCPM Eligible 12 (0.7) 15 (0.9) 18 (0.9) 
Not eligible 9 (2.0) 16 (2.2) 20 (1.6) 


NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. WCPM is an abbreviation for words correct per minute. About 2 percent of the 
students lacked valid information about their National School Lunch Program (NSLP) eligibility. These students are not reported by NAEP reading achievement level because of 
small sample size, which did not meet the NAEP reporting standard for robust estimation. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2018 Oral 
Reading Fluency study. 
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Table C-19. Estimates and standard errors for average oral reading performance by below NAEP Basic subgroups 
and English Language Learner (ELL) status: 2018 


a 

_ 

oO 

oy 

a 

S 

= 

a 

9 

S 

a 

2. 

a 

is 

= below below below 

iS ELL NAEP Basic| NAEP Basic| NAEP Basic 

2 Measure ORF variable status Low i 

z Oral reading fluency Passage reading words correct per minute Yes 69 (3.5) 95 (4.2) 107 (4.8) 

| (WCPM) No 71.235) 95 (2.4) 109 (2.8) 

oO 

a Passage reading rate: Words per minute Yes 84 (3.3) 104 (3.9) 115 (4.7) 

~ 

2) No 86 (2.6) 103 (2.4) 115 (2.8) 

igs] 

ez Passage reading accuracy: Percentage of Yes 81 (2.2) 91 (1.8) 93 (1.5) 

a words read correctly No 83 (1.3) 92 (0.7) 94 (0.6) 

B Passage reading expression Yes 2.6 (0.13) 3.3 (0.14) 3:7 (0.13) 

z No 2.7 (0.09) 3.4 (0.08) 3.8 (0.07) 

a. Word reading Word list reading: WCPM Yes 35 (2.3) 45 (2.6) 49 (3.0) 

oT No 34 (1.9) 45 (1.7) 50 (1.5) 

S Phonological decoding Pseudoword list reading: WCPM Yes 12 (1.0) 16 (1.4) 19 (2.1) 
No 11 (0.9) 16 (1.0) 19 (0.9) 


NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. WCPM is an abbreviation for words correct per minute. The results for 
English language learners are based on students who were assessed and cannot be generalized to the total population of such students. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2018 
Oral Reading Fluency study. 


Table C-20. Estimates and standard errors for average oral reading performance by below NAEP Basic subgroups and 
students with disabilities status: 2018 Fi 


below below 


Disability | NAEP Basic| NAEP Basic) NAEP Basic 


Measure ORF variable 


Oral reading fluency — Passage reading words correct per minute Yes 64 (3.2) 91 (5.0) 103 (6.8) 
(WCPM) No 75 (2.7) 96 (2.3) 109 (2.6) 

Passage reading rate: Words per minute Yes 81 (3.4) 100 (4.9) 110 (6.4) 

No 89 (2.9) 104 (2.4) 115 (2.7) 

Passage reading accuracy: Percentage of Yes 80 (1.6) 90 (1.7) 94 (1.9) 

words read correctly No 84 (1.4) 92 (08) 94 (0.5) 

Passage reading expression Yes 2.4 (0.15) 3.3 (0.16) 3.7 (0.17) 

No 2.8 (0.09) 3.4 (0.07) 3.8 (0.07) 

Word reading Word list reading: WCPM Yes a0 (2.7) 42 (3.9) 46 (4.0) 
No 37 (1.7) 46 (1.6) 50 (1.4) 

Phonological decoding Pseudoword list reading: WCPM Yes 9 (1.0) 14 (1.7) 17. (2.4) 
No 13 (1.0) 16 (0.9) 19 (0.8) 


NOTE: Pseudoword is a made-up but pronounceable word. Standard errors are in parentheses. WCPM is an abbreviation for words correct per minute. The results for students 
with disabilities are based on students who were assessed and cannot be generalized to the total population of such students. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2018 Oral 
Reading Fluency study. 
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